2023-11-24T05:58:05.719 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-24T05:58:05.719 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552 --owner scheduled_rishabh@teuthology --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} --verbose --name rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi -- /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552/orig.config.yaml 2023-11-24T05:58:05.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T05:58:05.942 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552 branch: rishabh-mgr-vol-misc description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} email: null first_in_suite: false job_id: '7465552' kernel: flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-mgr-vol-misc ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: rishabh-mgr-vol-misc sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3305 sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c sleep_before_teardown: 0 suite: fs:volumes suite_branch: rishabh-mgr-vol-misc suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHxC54AoB40hHZJBei+C+H1tj0i0mbvv+5ZiCTjBje1ZITrJ1meyYKiQ7HmT71SkrxFCLB1mDtonwt8WAVmum/Q= smithi006.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIKjSSSw3SKfP6l+THmyThcZ6hDVcLI75y1IghvDKxCpc9aPaHYHE7irCADEi8Q+VWGwhXLYcR4sptlmUvccwM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-24_05:41:33 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-24T05:58:05.944 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa; will attempt to use it 2023-11-24T05:58:05.944 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks 2023-11-24T05:58:05.945 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-24T05:58:05.947 INFO:teuthology.task.internal:Checking packages... 2023-11-24T05:58:05.975 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a13ea5ff00cb4df7a9cc941166ddeb358881144c' 2023-11-24T05:58:05.975 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-24T05:58:05.975 INFO:teuthology.packaging:ref: None 2023-11-24T05:58:05.975 INFO:teuthology.packaging:tag: None 2023-11-24T05:58:05.975 INFO:teuthology.packaging:branch: rishabh-mgr-vol-misc 2023-11-24T05:58:05.975 INFO:teuthology.packaging:sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T05:58:05.976 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-mgr-vol-misc 2023-11-24T05:58:06.129 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7484.ga13ea5ff 2023-11-24T05:58:06.129 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-24T05:58:06.138 INFO:teuthology.task.internal:no buildpackages task found 2023-11-24T05:58:06.138 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-24T05:58:06.147 INFO:teuthology.task.internal:Saving configuration 2023-11-24T05:58:06.173 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-24T05:58:06.188 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T05:58:06.221 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 05:53:28.331729', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHxC54AoB40hHZJBei+C+H1tj0i0mbvv+5ZiCTjBje1ZITrJ1meyYKiQ7HmT71SkrxFCLB1mDtonwt8WAVmum/Q='} 2023-11-24T05:58:06.247 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 05:53:28.329960', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIKjSSSw3SKfP6l+THmyThcZ6hDVcLI75y1IghvDKxCpc9aPaHYHE7irCADEi8Q+VWGwhXLYcR4sptlmUvccwM='} 2023-11-24T05:58:06.248 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-24T05:58:06.254 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-24T05:58:06.254 INFO:teuthology.task.internal:roles: ubuntu@smithi006.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-24T05:58:06.255 INFO:teuthology.run_tasks:Running task console_log... 2023-11-24T05:58:06.425 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd91fa4eb80>, signals=[15]) 2023-11-24T05:58:06.426 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-24T05:58:06.432 INFO:teuthology.task.internal:Opening connections... 2023-11-24T05:58:06.432 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T05:58:06.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T05:58:06.537 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T05:58:06.537 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T05:58:06.634 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-24T05:58:06.648 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2023-11-24T05:58:06.667 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2023-11-24T05:58:06.667 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2023-11-24T05:58:06.725 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream" 2023-11-24T05:58:06.726 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="9" 2023-11-24T05:58:06.727 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos" 2023-11-24T05:58:06.727 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora" 2023-11-24T05:58:06.727 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="9" 2023-11-24T05:58:06.727 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el9" 2023-11-24T05:58:06.727 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-24T05:58:06.728 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2023-11-24T05:58:06.728 INFO:teuthology.orchestra.run.smithi005.stdout:LOGO="fedora-logo-icon" 2023-11-24T05:58:06.728 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-24T05:58:06.728 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/" 2023-11-24T05:58:06.728 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T05:58:06.729 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-24T05:58:06.729 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-24T05:58:06.729 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2023-11-24T05:58:06.765 DEBUG:teuthology.orchestra.run.smithi006:> uname -m 2023-11-24T05:58:06.783 INFO:teuthology.orchestra.run.smithi006.stdout:x86_64 2023-11-24T05:58:06.784 DEBUG:teuthology.orchestra.run.smithi006:> cat /etc/os-release 2023-11-24T05:58:06.842 INFO:teuthology.orchestra.run.smithi006.stdout:NAME="CentOS Stream" 2023-11-24T05:58:06.842 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION="9" 2023-11-24T05:58:06.842 INFO:teuthology.orchestra.run.smithi006.stdout:ID="centos" 2023-11-24T05:58:06.842 INFO:teuthology.orchestra.run.smithi006.stdout:ID_LIKE="rhel fedora" 2023-11-24T05:58:06.842 INFO:teuthology.orchestra.run.smithi006.stdout:VERSION_ID="9" 2023-11-24T05:58:06.842 INFO:teuthology.orchestra.run.smithi006.stdout:PLATFORM_ID="platform:el9" 2023-11-24T05:58:06.843 INFO:teuthology.orchestra.run.smithi006.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-24T05:58:06.843 INFO:teuthology.orchestra.run.smithi006.stdout:ANSI_COLOR="0;31" 2023-11-24T05:58:06.843 INFO:teuthology.orchestra.run.smithi006.stdout:LOGO="fedora-logo-icon" 2023-11-24T05:58:06.843 INFO:teuthology.orchestra.run.smithi006.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-24T05:58:06.843 INFO:teuthology.orchestra.run.smithi006.stdout:HOME_URL="https://centos.org/" 2023-11-24T05:58:06.843 INFO:teuthology.orchestra.run.smithi006.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T05:58:06.843 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-24T05:58:06.843 INFO:teuthology.orchestra.run.smithi006.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-24T05:58:06.843 INFO:teuthology.lock.ops:Updating smithi006.front.sepia.ceph.com on lock server 2023-11-24T05:58:06.874 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-24T05:58:06.887 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-24T05:58:06.893 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-24T05:58:06.893 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2023-11-24T05:58:06.896 DEBUG:teuthology.orchestra.run.smithi006:> test '!' -e /home/ubuntu/cephtest 2023-11-24T05:58:06.913 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-24T05:58:06.921 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-24T05:58:06.921 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2023-11-24T05:58:06.954 DEBUG:teuthology.orchestra.run.smithi006:> test -z $(ls -A /var/lib/ceph) 2023-11-24T05:58:06.985 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-24T05:58:07.050 INFO:teuthology.run_tasks:Running task kernel... 2023-11-24T05:58:07.073 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-24T05:58:07.073 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, timeout 300 2023-11-24T05:58:07.073 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T05:58:07.074 DEBUG:teuthology.orchestra.run.smithi006:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T05:58:07.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T05:58:07.092 DEBUG:teuthology.task.kernel:sha1 for mon.a is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:58:07.092 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-24T05:58:07.092 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2023-11-24T05:58:07.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T05:58:07.097 DEBUG:teuthology.task.kernel:sha1 for mon.b is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:58:07.097 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-24T05:58:07.097 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2023-11-24T05:58:07.150 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-361.el9.x86_64 2023-11-24T05:58:07.150 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:58:07.152 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-24T05:58:07.152 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:58:07.158 INFO:teuthology.orchestra.run.smithi006.stdout:5.14.0-361.el9.x86_64 2023-11-24T05:58:07.158 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:58:07.158 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-24T05:58:07.158 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:58:07.306 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum upgrade -y linux-firmware 2023-11-24T05:58:07.432 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum upgrade -y linux-firmware 2023-11-24T05:58:09.489 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS-9 - AppStream 12 MB/s | 6.4 MB 00:00 2023-11-24T05:58:09.757 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - AppStream 11 MB/s | 6.4 MB 00:00 2023-11-24T05:58:10.930 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS-9 - BaseOS 5.8 MB/s | 2.0 MB 00:00 2023-11-24T05:58:11.257 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - BaseOS 4.9 MB/s | 2.0 MB 00:00 2023-11-24T05:58:12.221 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 8.4 MB/s | 7.9 MB 00:00 2023-11-24T05:58:14.056 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 28 MB/s | 18 MB 00:00 2023-11-24T05:58:14.202 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 7.9 MB 00:02 2023-11-24T05:58:19.548 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 5.8 MB 00:01 2023-11-24T05:58:20.079 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - AppStream 3.8 MB/s | 18 MB 00:04 2023-11-24T05:58:21.165 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 15 kB 00:00 2023-11-24T05:58:21.917 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 33 MB/s | 20 MB 00:00 2023-11-24T05:58:24.032 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - CRB 14 MB/s | 5.8 MB 00:00 2023-11-24T05:58:25.350 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - Extras packages 90 kB/s | 15 kB 00:00 2023-11-24T05:58:26.485 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 20 MB/s | 20 MB 00:00 2023-11-24T05:58:26.825 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2023-11-24T05:58:29.318 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2023-11-24T05:58:29.325 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2023-11-24T05:58:29.325 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2023-11-24T05:58:29.325 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2023-11-24T05:58:29.325 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2023-11-24T05:58:29.325 INFO:teuthology.orchestra.run.smithi005.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-11-24T05:58:29.325 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T05:58:29.325 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2023-11-24T05:58:29.326 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2023-11-24T05:58:29.326 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 1 Package 2023-11-24T05:58:29.326 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T05:58:29.326 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 328 M 2023-11-24T05:58:29.326 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2023-11-24T05:58:31.387 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2023-11-24T05:58:33.938 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-11-24T05:58:33.951 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-11-24T05:58:33.952 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-11-24T05:58:33.952 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-11-24T05:58:33.952 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrading: 2023-11-24T05:58:33.952 INFO:teuthology.orchestra.run.smithi006.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-11-24T05:58:33.952 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T05:58:33.952 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-11-24T05:58:33.952 INFO:teuthology.orchestra.run.smithi006.stdout:================================================================================ 2023-11-24T05:58:33.952 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrade 1 Package 2023-11-24T05:58:33.952 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T05:58:33.952 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 328 M 2023-11-24T05:58:33.952 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-11-24T05:58:34.451 INFO:teuthology.orchestra.run.smithi005.stdout:linux-firmware-20231030-141.el9.noarch.rpm 64 MB/s | 328 MB 00:05 2023-11-24T05:58:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2023-11-24T05:58:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:Total 64 MB/s | 328 MB 00:05 2023-11-24T05:58:34.452 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2023-11-24T05:58:34.548 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2023-11-24T05:58:34.548 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2023-11-24T05:58:36.731 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2023-11-24T05:58:36.732 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2023-11-24T05:58:38.952 INFO:teuthology.orchestra.run.smithi006.stdout:linux-firmware-20231030-141.el9.noarch.rpm 66 MB/s | 328 MB 00:05 2023-11-24T05:58:38.954 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-11-24T05:58:38.954 INFO:teuthology.orchestra.run.smithi006.stdout:Total 66 MB/s | 328 MB 00:05 2023-11-24T05:58:38.954 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-11-24T05:58:39.042 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-11-24T05:58:39.043 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-11-24T05:58:41.269 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-11-24T05:58:41.269 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-11-24T05:58:42.629 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2023-11-24T05:58:43.085 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T05:58:43.159 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T05:58:44.145 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T05:58:44.145 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T05:58:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T05:58:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T05:58:44.536 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2023-11-24T05:58:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: linux-firmware-20231030-141.el9.noarch 2023-11-24T05:58:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T05:58:44.537 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2023-11-24T05:58:44.823 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.a... 2023-11-24T05:58:44.824 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:58:44.965 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ 2023-11-24T05:58:44.966 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-24T05:58:44.968 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi005.front.sepia.ceph.com to complete... 2023-11-24T05:58:47.190 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-11-24T05:58:47.466 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T05:58:47.562 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T05:58:48.536 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T05:58:48.537 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T05:58:48.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T05:58:48.965 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T05:58:48.965 INFO:teuthology.orchestra.run.smithi006.stdout:Upgraded: 2023-11-24T05:58:48.965 INFO:teuthology.orchestra.run.smithi006.stdout: linux-firmware-20231030-141.el9.noarch 2023-11-24T05:58:48.965 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T05:58:48.965 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-11-24T05:58:49.247 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.b... 2023-11-24T05:58:49.248 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T05:58:49.468 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ 2023-11-24T05:58:49.468 DEBUG:teuthology.orchestra.run.smithi006:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-24T05:58:49.471 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi006.front.sepia.ceph.com to complete... 2023-11-24T05:59:02.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24 05:59:02 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [320493731/320493731] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-24T05:59:02.149 INFO:teuthology.orchestra.run.smithi005.stderr:FINISHED --2023-11-24 05:59:02-- 2023-11-24T05:59:02.149 INFO:teuthology.orchestra.run.smithi005.stderr:Total wall clock time: 17s 2023-11-24T05:59:02.149 INFO:teuthology.orchestra.run.smithi005.stderr:Downloaded: 1 files, 306M in 17s (18.1 MB/s) 2023-11-24T05:59:02.151 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.a... 2023-11-24T05:59:02.151 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-24T05:59:04.093 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying... ######################################## 2023-11-24T05:59:04.149 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing... ######################################## 2023-11-24T05:59:04.233 INFO:teuthology.orchestra.run.smithi005.stdout:Updating / installing... 2023-11-24T05:59:06.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24 05:59:06 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [320493731/320493731] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-24T05:59:06.355 INFO:teuthology.orchestra.run.smithi006.stderr:FINISHED --2023-11-24 05:59:06-- 2023-11-24T05:59:06.355 INFO:teuthology.orchestra.run.smithi006.stderr:Total wall clock time: 17s 2023-11-24T05:59:06.355 INFO:teuthology.orchestra.run.smithi006.stderr:Downloaded: 1 files, 306M in 17s (18.5 MB/s) 2023-11-24T05:59:06.357 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on mon.b... 2023-11-24T05:59:06.357 DEBUG:teuthology.orchestra.run.smithi006:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-24T05:59:08.301 INFO:teuthology.orchestra.run.smithi006.stdout:Verifying... ######################################## 2023-11-24T05:59:08.356 INFO:teuthology.orchestra.run.smithi006.stdout:Preparing... ######################################## 2023-11-24T05:59:08.440 INFO:teuthology.orchestra.run.smithi006.stdout:Updating / installing... 2023-11-24T05:59:11.592 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-24T05:59:15.809 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-24T05:59:26.843 INFO:teuthology.orchestra.run.smithi005.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-24T05:59:31.044 INFO:teuthology.orchestra.run.smithi006.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-24T05:59:48.778 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-24T05:59:48.779 DEBUG:teuthology.orchestra.run.smithi005:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-24T05:59:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T05:59:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-24T05:59:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-24T05:59:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-24T05:59:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-24T05:59:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-24T05:59:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-24T05:59:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-24T05:59:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-24T05:59:48.888 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-24T05:59:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-24T05:59:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-24T05:59:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-24T05:59:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-24T05:59:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-24T05:59:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-24T05:59:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-24T05:59:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-24T05:59:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-24T05:59:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-24T05:59:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-24T05:59:48.889 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-24T05:59:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-24T05:59:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-24T05:59:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-24T05:59:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-24T05:59:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-24T05:59:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-24T05:59:48.890 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-24T05:59:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-24T05:59:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-24T05:59:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-24T05:59:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-24T05:59:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-24T05:59:48.895 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-24T05:59:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-24T05:59:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-24T05:59:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-24T05:59:48.896 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-24T05:59:48.897 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-24T05:59:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-24T05:59:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-24T05:59:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-24T05:59:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-24T05:59:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-24T05:59:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-24T05:59:48.898 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-24T05:59:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-24T05:59:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-24T05:59:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-24T05:59:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-24T05:59:48.899 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-24T05:59:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-24T05:59:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-24T05:59:48.900 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-24T05:59:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-24T05:59:48.926 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-24T05:59:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-24T05:59:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-24T05:59:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-24T05:59:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-24T05:59:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-24T05:59:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-24T05:59:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-24T05:59:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-24T05:59:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-24T05:59:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-24T05:59:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-24T05:59:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-24T05:59:48.928 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-24T05:59:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-24T05:59:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-24T05:59:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-24T05:59:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-24T05:59:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-24T05:59:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-24T05:59:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-24T05:59:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-24T05:59:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-24T05:59:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-24T05:59:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-24T05:59:48.929 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-24T05:59:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-24T05:59:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-24T05:59:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-24T05:59:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-24T05:59:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-24T05:59:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-24T05:59:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-24T05:59:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-24T05:59:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-24T05:59:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-24T05:59:48.930 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-24T05:59:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-24T05:59:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-24T05:59:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-24T05:59:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-24T05:59:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-24T05:59:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-24T05:59:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-24T05:59:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-24T05:59:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-24T05:59:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-24T05:59:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-24T05:59:48.931 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-24T05:59:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-24T05:59:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-24T05:59:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-24T05:59:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-24T05:59:48.932 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-24T05:59:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-24T05:59:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-24T05:59:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-24T05:59:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-24T05:59:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-24T05:59:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-24T05:59:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-24T05:59:48.934 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-24T05:59:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-24T05:59:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-24T05:59:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-24T05:59:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-24T05:59:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-24T05:59:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-24T05:59:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-24T05:59:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-24T05:59:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-24T05:59:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-24T05:59:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-24T05:59:48.935 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-24T05:59:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-24T05:59:48.936 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-24T05:59:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-24T05:59:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-24T05:59:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-24T05:59:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-24T05:59:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-24T05:59:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-24T05:59:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-24T05:59:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-24T05:59:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-24T05:59:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-24T05:59:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-24T05:59:48.937 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-24T05:59:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-24T05:59:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-24T05:59:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-24T05:59:48.938 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-24T05:59:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-24T05:59:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-24T05:59:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-24T05:59:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-24T05:59:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-24T05:59:48.943 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-24T05:59:48.948 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-24T05:59:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-24T05:59:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-24T05:59:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-24T05:59:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-24T05:59:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-24T05:59:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-24T05:59:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-24T05:59:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-24T05:59:48.949 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-24T05:59:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-24T05:59:48.950 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-24T05:59:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-24T05:59:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-24T05:59:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-24T05:59:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-24T05:59:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-24T05:59:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-24T05:59:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-24T05:59:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-24T05:59:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-24T05:59:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-24T05:59:48.951 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-24T05:59:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-24T05:59:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-24T05:59:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-24T05:59:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-24T05:59:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-24T05:59:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-24T05:59:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-24T05:59:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-24T05:59:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-24T05:59:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-24T05:59:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-24T05:59:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-24T05:59:48.952 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-24T05:59:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-24T05:59:48.953 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-24T05:59:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-24T05:59:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-24T05:59:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-24T05:59:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-24T05:59:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-24T05:59:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-24T05:59:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-24T05:59:48.954 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-24T05:59:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-24T05:59:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-24T05:59:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-24T05:59:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-24T05:59:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-24T05:59:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-24T05:59:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-24T05:59:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-24T05:59:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-24T05:59:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-24T05:59:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-24T05:59:48.955 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-24T05:59:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-24T05:59:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-24T05:59:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-24T05:59:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-24T05:59:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-24T05:59:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-24T05:59:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-24T05:59:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-24T05:59:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-24T05:59:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-24T05:59:48.956 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-24T05:59:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-24T05:59:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-24T05:59:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-24T05:59:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-24T05:59:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-24T05:59:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-24T05:59:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-24T05:59:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-24T05:59:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-24T05:59:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-24T05:59:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-24T05:59:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-24T05:59:48.957 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-24T05:59:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-24T05:59:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-24T05:59:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-24T05:59:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-24T05:59:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-24T05:59:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-24T05:59:48.958 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-24T05:59:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-24T05:59:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-24T05:59:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-24T05:59:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-24T05:59:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-24T05:59:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-24T05:59:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-24T05:59:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-24T05:59:48.960 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-24T05:59:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-24T05:59:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-24T05:59:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-24T05:59:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-24T05:59:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-24T05:59:48.964 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-24T05:59:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-24T05:59:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-24T05:59:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-24T05:59:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-24T05:59:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-24T05:59:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-24T05:59:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-24T05:59:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-24T05:59:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-24T05:59:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-24T05:59:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-24T05:59:48.965 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-24T05:59:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-24T05:59:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-24T05:59:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-24T05:59:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-24T05:59:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-24T05:59:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-24T05:59:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-24T05:59:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-24T05:59:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-24T05:59:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-24T05:59:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-24T05:59:48.966 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-24T05:59:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-24T05:59:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-24T05:59:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-24T05:59:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-24T05:59:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-24T05:59:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-24T05:59:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-24T05:59:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-24T05:59:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-24T05:59:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-24T05:59:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-24T05:59:48.967 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-24T05:59:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-24T05:59:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-24T05:59:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-24T05:59:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-24T05:59:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-24T05:59:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-24T05:59:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-24T05:59:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-24T05:59:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-24T05:59:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-24T05:59:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-24T05:59:48.968 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-24T05:59:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-24T05:59:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-24T05:59:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-24T05:59:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-24T05:59:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-24T05:59:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-24T05:59:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-24T05:59:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-24T05:59:48.978 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-24T05:59:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-24T05:59:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-24T05:59:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-24T05:59:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-24T05:59:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-24T05:59:48.979 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-24T05:59:48.980 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-24T05:59:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-24T05:59:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-24T05:59:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-24T05:59:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-24T05:59:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-24T05:59:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-24T05:59:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-24T05:59:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-24T05:59:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-24T05:59:48.981 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-24T05:59:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-24T05:59:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-24T05:59:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-24T05:59:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-24T05:59:48.982 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-24T05:59:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-24T05:59:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-24T05:59:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-24T05:59:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-24T05:59:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-24T05:59:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-24T05:59:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-24T05:59:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-24T05:59:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-24T05:59:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-24T05:59:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-24T05:59:48.983 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-24T05:59:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-24T05:59:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-24T05:59:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-24T05:59:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-24T05:59:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-24T05:59:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-24T05:59:48.984 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-24T05:59:48.990 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-24T05:59:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-24T05:59:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-24T05:59:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-24T05:59:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-24T05:59:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-24T05:59:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-24T05:59:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-24T05:59:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-24T05:59:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-24T05:59:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-24T05:59:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-24T05:59:48.991 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-24T05:59:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-24T05:59:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-24T05:59:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-24T05:59:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-24T05:59:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-24T05:59:48.992 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-24T05:59:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-24T05:59:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-24T05:59:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-24T05:59:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-24T05:59:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-24T05:59:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-24T05:59:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-24T05:59:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-24T05:59:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-24T05:59:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-24T05:59:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-24T05:59:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-24T05:59:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-24T05:59:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-24T05:59:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-24T05:59:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-24T05:59:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-24T05:59:48.994 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-24T05:59:48.999 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-24T05:59:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-24T05:59:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-24T05:59:49.000 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-24T05:59:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-24T05:59:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-24T05:59:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-24T05:59:49.002 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-24T05:59:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-24T05:59:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-24T05:59:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-24T05:59:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-24T05:59:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-24T05:59:49.003 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-24T05:59:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-24T05:59:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-24T05:59:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-24T05:59:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-24T05:59:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-24T05:59:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-24T05:59:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-24T05:59:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-24T05:59:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-24T05:59:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-24T05:59:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-24T05:59:49.004 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-24T05:59:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-24T05:59:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-24T05:59:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-24T05:59:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-24T05:59:49.005 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-24T05:59:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-24T05:59:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-24T05:59:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-24T05:59:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-24T05:59:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-24T05:59:49.006 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-24T05:59:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-24T05:59:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-24T05:59:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-24T05:59:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-24T05:59:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-24T05:59:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-24T05:59:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-24T05:59:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-24T05:59:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-24T05:59:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-24T05:59:49.007 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-24T05:59:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-24T05:59:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-24T05:59:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-24T05:59:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-24T05:59:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-24T05:59:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-24T05:59:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-24T05:59:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-24T05:59:49.009 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-24T05:59:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-24T05:59:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-24T05:59:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-24T05:59:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-24T05:59:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-24T05:59:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-24T05:59:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-24T05:59:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-24T05:59:49.010 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-24T05:59:49.016 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-24T05:59:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-24T05:59:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-24T05:59:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-24T05:59:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-24T05:59:49.017 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-24T05:59:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-24T05:59:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-24T05:59:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-24T05:59:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-24T05:59:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-24T05:59:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-24T05:59:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-24T05:59:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-24T05:59:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-24T05:59:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-24T05:59:49.019 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-24T05:59:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-24T05:59:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-24T05:59:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-24T05:59:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-24T05:59:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-24T05:59:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-24T05:59:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-24T05:59:49.021 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-24T05:59:49.022 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-24T05:59:49.022 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-24T05:59:49.022 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-24T05:59:49.022 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-24T05:59:49.023 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-24T05:59:49.023 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-24T05:59:49.023 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-24T05:59:49.023 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-24T05:59:49.023 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-24T05:59:49.023 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-24T05:59:49.024 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-24T05:59:49.024 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-24T05:59:49.024 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-24T05:59:49.025 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-24T05:59:49.026 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-24T05:59:49.027 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-24T05:59:49.027 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-24T05:59:49.027 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-24T05:59:49.027 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-24T05:59:49.027 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-24T05:59:49.028 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-24T05:59:49.028 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-24T05:59:49.028 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-24T05:59:49.028 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-24T05:59:49.028 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-24T05:59:49.028 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-24T05:59:49.029 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-24T05:59:49.029 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-24T05:59:49.029 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-24T05:59:49.029 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-24T05:59:49.029 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-24T05:59:49.030 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-24T05:59:49.030 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-24T05:59:49.030 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-24T05:59:49.030 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-24T05:59:49.030 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-24T05:59:49.030 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-24T05:59:49.031 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-24T05:59:49.031 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-24T05:59:49.031 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-24T05:59:49.032 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-24T05:59:49.035 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-24T05:59:49.035 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-24T05:59:49.035 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-24T05:59:49.036 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-24T05:59:49.036 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-24T05:59:49.036 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-24T05:59:49.036 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-24T05:59:49.036 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-24T05:59:49.037 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-24T05:59:49.037 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-24T05:59:49.037 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-24T05:59:49.037 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-24T05:59:49.038 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-24T05:59:49.038 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-24T05:59:49.038 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-24T05:59:49.038 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-24T05:59:49.038 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-24T05:59:49.039 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-24T05:59:49.039 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-24T05:59:49.039 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-24T05:59:49.039 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-24T05:59:49.039 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-24T05:59:49.039 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-24T05:59:49.040 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-24T05:59:49.040 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-24T05:59:49.040 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-24T05:59:49.040 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-24T05:59:49.040 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-24T05:59:49.041 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-24T05:59:49.041 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-24T05:59:49.041 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-24T05:59:49.041 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-24T05:59:49.041 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-24T05:59:49.041 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-24T05:59:49.042 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-24T05:59:49.042 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-24T05:59:49.042 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-24T05:59:49.043 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-24T05:59:49.043 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-24T05:59:49.043 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-24T05:59:49.043 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-24T05:59:49.043 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-24T05:59:49.043 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-24T05:59:49.044 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-24T05:59:49.044 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-24T05:59:49.044 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-24T05:59:49.044 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-24T05:59:49.044 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-24T05:59:49.055 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-24T05:59:49.062 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-24T05:59:49.062 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-24T05:59:49.062 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-24T05:59:49.062 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-24T05:59:49.063 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-24T05:59:49.064 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-24T05:59:49.064 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-24T05:59:49.064 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-24T05:59:49.064 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-24T05:59:49.064 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-24T05:59:49.065 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-24T05:59:49.065 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-24T05:59:49.067 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-24T05:59:49.067 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-24T05:59:49.067 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-24T05:59:49.067 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-24T05:59:49.068 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-24T05:59:49.068 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-24T05:59:49.068 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-24T05:59:49.068 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-24T05:59:49.069 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-24T05:59:49.069 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-24T05:59:49.069 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-24T05:59:49.069 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-24T05:59:49.069 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-24T05:59:49.070 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-24T05:59:49.070 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-24T05:59:49.070 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-24T05:59:49.070 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-24T05:59:49.070 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-24T05:59:49.071 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-24T05:59:49.071 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-24T05:59:49.071 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-24T05:59:49.071 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-24T05:59:49.075 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-24T05:59:49.075 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-24T05:59:49.076 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-24T05:59:49.076 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-24T05:59:49.076 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-24T05:59:49.076 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-24T05:59:49.077 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-24T05:59:49.077 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-24T05:59:49.077 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-24T05:59:49.077 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-24T05:59:49.077 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-24T05:59:49.078 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-24T05:59:49.078 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-24T05:59:49.078 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-24T05:59:49.078 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-24T05:59:49.078 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-24T05:59:49.079 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-24T05:59:49.079 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-24T05:59:49.079 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-24T05:59:49.079 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-24T05:59:49.079 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-24T05:59:49.079 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-24T05:59:49.079 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-24T05:59:49.079 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-24T05:59:49.079 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-24T05:59:49.080 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-24T05:59:49.080 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-24T05:59:49.080 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-24T05:59:49.080 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-24T05:59:49.080 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-24T05:59:49.080 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-24T05:59:49.080 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-24T05:59:49.080 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-24T05:59:49.080 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-24T05:59:49.080 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-24T05:59:49.080 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-24T05:59:49.081 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-24T05:59:49.081 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-24T05:59:49.081 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-24T05:59:49.081 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-24T05:59:49.081 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-24T05:59:49.081 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-24T05:59:49.082 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-24T05:59:49.082 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-24T05:59:49.082 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-24T05:59:49.082 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-24T05:59:49.082 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-24T05:59:49.082 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-24T05:59:49.082 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-24T05:59:49.082 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-24T05:59:49.082 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-24T05:59:49.082 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-24T05:59:49.082 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-24T05:59:49.083 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-24T05:59:49.083 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-24T05:59:49.083 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-24T05:59:49.083 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-24T05:59:49.083 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-24T05:59:49.083 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-24T05:59:49.084 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-24T05:59:49.084 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-24T05:59:49.084 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-24T05:59:49.084 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-24T05:59:49.084 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-24T05:59:49.084 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-24T05:59:49.084 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-24T05:59:49.084 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-24T05:59:49.084 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-24T05:59:49.084 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-24T05:59:49.084 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-24T05:59:49.084 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-24T05:59:49.085 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-24T05:59:49.085 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-24T05:59:49.085 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-24T05:59:49.085 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-24T05:59:49.085 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-24T05:59:49.085 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-24T05:59:49.086 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-24T05:59:49.086 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-24T05:59:49.086 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-24T05:59:49.086 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-24T05:59:49.086 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-24T05:59:49.086 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-24T05:59:49.086 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-24T05:59:49.086 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-24T05:59:49.086 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-24T05:59:49.086 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-24T05:59:49.086 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-24T05:59:49.087 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-24T05:59:49.087 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-24T05:59:49.087 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-24T05:59:49.087 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-24T05:59:49.087 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-24T05:59:49.087 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-24T05:59:49.087 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-24T05:59:49.087 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-24T05:59:49.087 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-24T05:59:49.087 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-24T05:59:49.087 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-24T05:59:49.087 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-24T05:59:49.088 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-24T05:59:49.088 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-24T05:59:49.088 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-24T05:59:49.088 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-24T05:59:49.088 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-24T05:59:49.088 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-24T05:59:49.088 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-24T05:59:49.088 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-24T05:59:49.088 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-24T05:59:49.088 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-24T05:59:49.088 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-24T05:59:49.089 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-24T05:59:49.089 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-24T05:59:49.089 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-24T05:59:49.089 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-24T05:59:49.089 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-24T05:59:49.089 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-24T05:59:49.089 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-24T05:59:49.089 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-24T05:59:49.089 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-24T05:59:49.089 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-24T05:59:49.089 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-24T05:59:49.090 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-24T05:59:49.090 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-24T05:59:49.090 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-24T05:59:49.090 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-24T05:59:49.090 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-24T05:59:49.090 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-24T05:59:49.090 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-24T05:59:49.090 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-24T05:59:49.090 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-24T05:59:49.090 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-24T05:59:49.090 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-24T05:59:49.091 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-24T05:59:49.091 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-24T05:59:49.091 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-24T05:59:49.091 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-24T05:59:49.091 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-24T05:59:49.091 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-24T05:59:49.091 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-24T05:59:49.091 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-24T05:59:49.091 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-24T05:59:49.091 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-24T05:59:49.091 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-24T05:59:49.091 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-24T05:59:49.092 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-24T05:59:49.092 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-24T05:59:49.092 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-24T05:59:49.092 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-24T05:59:49.092 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-24T05:59:49.092 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-24T05:59:49.092 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-24T05:59:49.092 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-24T05:59:49.092 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-24T05:59:49.092 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-24T05:59:49.092 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-24T05:59:49.093 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-24T05:59:49.093 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-24T05:59:49.093 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-24T05:59:49.093 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-24T05:59:49.093 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-24T05:59:49.093 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-24T05:59:49.094 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-24T05:59:49.094 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-24T05:59:49.094 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-24T05:59:49.094 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-24T05:59:49.094 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-24T05:59:49.094 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-24T05:59:49.094 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-24T05:59:49.094 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-24T05:59:49.095 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-24T05:59:49.095 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-24T05:59:49.095 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-24T05:59:49.095 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-24T05:59:49.095 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-24T05:59:49.095 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-24T05:59:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-24T05:59:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-24T05:59:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-24T05:59:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-24T05:59:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-24T05:59:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-24T05:59:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-24T05:59:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-24T05:59:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-24T05:59:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-24T05:59:49.096 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-24T05:59:49.097 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-24T05:59:49.097 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-24T05:59:49.097 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-24T05:59:49.097 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-24T05:59:49.097 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-24T05:59:49.097 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-24T05:59:49.097 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-24T05:59:49.097 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-24T05:59:49.097 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-24T05:59:49.097 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-24T05:59:49.097 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-24T05:59:49.098 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-24T05:59:49.098 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-24T05:59:49.098 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-24T05:59:49.098 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-24T05:59:49.098 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-24T05:59:49.098 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-24T05:59:49.098 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-24T05:59:49.098 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-24T05:59:49.098 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-24T05:59:49.098 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-24T05:59:49.098 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-24T05:59:49.098 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-24T05:59:49.099 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-24T05:59:49.099 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-24T05:59:49.099 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-24T05:59:49.099 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-24T05:59:49.099 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-24T05:59:49.099 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-24T05:59:49.099 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-24T05:59:49.099 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-24T05:59:49.099 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-24T05:59:49.099 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-24T05:59:49.099 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-24T05:59:49.100 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-24T05:59:49.100 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-24T05:59:49.100 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-24T05:59:49.100 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-24T05:59:49.100 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-24T05:59:49.100 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-24T05:59:49.100 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-24T05:59:49.100 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-24T05:59:49.100 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-24T05:59:49.100 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-24T05:59:49.100 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-24T05:59:49.100 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-24T05:59:49.101 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-24T05:59:49.101 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-24T05:59:49.101 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-24T05:59:49.101 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-24T05:59:49.101 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-24T05:59:49.101 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-24T05:59:49.101 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-24T05:59:49.101 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-24T05:59:49.101 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-24T05:59:49.101 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-24T05:59:49.101 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-24T05:59:49.102 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-24T05:59:49.102 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-24T05:59:49.102 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-24T05:59:49.102 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-24T05:59:49.102 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-24T05:59:49.102 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-24T05:59:49.102 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-24T05:59:49.102 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-24T05:59:49.102 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-24T05:59:49.102 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-24T05:59:49.102 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-24T05:59:49.103 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-24T05:59:49.103 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-24T05:59:49.103 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-24T05:59:49.103 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-24T05:59:49.103 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-24T05:59:49.103 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-24T05:59:49.103 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-24T05:59:49.103 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-24T05:59:49.103 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-24T05:59:49.103 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-24T05:59:49.103 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-24T05:59:49.104 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-24T05:59:49.104 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-24T05:59:49.104 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-24T05:59:49.104 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-24T05:59:49.104 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-24T05:59:49.104 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-24T05:59:49.104 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-24T05:59:49.105 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-24T05:59:49.105 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-24T05:59:49.105 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-24T05:59:49.105 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-24T05:59:49.105 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-24T05:59:49.105 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-24T05:59:49.105 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-24T05:59:49.105 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-24T05:59:49.105 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-24T05:59:49.105 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-24T05:59:49.105 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-24T05:59:49.106 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-24T05:59:49.106 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-24T05:59:49.106 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-24T05:59:49.106 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-24T05:59:49.106 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-24T05:59:49.106 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-24T05:59:49.106 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-24T05:59:49.107 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-24T05:59:49.107 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-24T05:59:49.107 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-24T05:59:49.107 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-24T05:59:49.107 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-24T05:59:49.107 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-24T05:59:49.107 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-24T05:59:49.107 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-24T05:59:49.107 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-24T05:59:49.107 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-24T05:59:49.107 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-24T05:59:49.107 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-24T05:59:49.108 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-24T05:59:49.108 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-24T05:59:49.108 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-24T05:59:49.108 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-24T05:59:49.108 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-24T05:59:49.108 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-24T05:59:49.108 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-24T05:59:49.108 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-24T05:59:49.108 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-24T05:59:49.108 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-24T05:59:49.108 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-24T05:59:49.109 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-24T05:59:49.109 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-24T05:59:49.109 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-24T05:59:49.109 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-24T05:59:49.109 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-24T05:59:49.109 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-24T05:59:49.109 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-24T05:59:49.109 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-24T05:59:49.109 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-24T05:59:49.109 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-24T05:59:49.109 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-24T05:59:49.109 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-24T05:59:49.110 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-24T05:59:49.110 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-24T05:59:49.110 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-24T05:59:49.110 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-24T05:59:49.110 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-24T05:59:49.110 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-24T05:59:49.110 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-24T05:59:49.110 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-24T05:59:49.110 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-24T05:59:49.110 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-24T05:59:49.110 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-24T05:59:49.111 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-24T05:59:49.111 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-24T05:59:49.111 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-24T05:59:49.111 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-24T05:59:49.111 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-24T05:59:49.111 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-24T05:59:49.111 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-24T05:59:49.111 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-24T05:59:49.111 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-24T05:59:49.111 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-24T05:59:49.111 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-24T05:59:49.111 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-24T05:59:49.112 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-24T05:59:49.112 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-24T05:59:49.112 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-24T05:59:49.112 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-24T05:59:49.112 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-24T05:59:49.112 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-24T05:59:49.112 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-24T05:59:49.112 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-24T05:59:49.112 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-24T05:59:49.112 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-24T05:59:49.112 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-24T05:59:49.113 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-24T05:59:49.113 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-24T05:59:49.113 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-24T05:59:49.113 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-24T05:59:49.113 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-24T05:59:49.113 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-24T05:59:49.113 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-24T05:59:49.113 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-24T05:59:49.113 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-24T05:59:49.113 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-24T05:59:49.113 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-24T05:59:49.113 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-24T05:59:49.114 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-24T05:59:49.114 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-24T05:59:49.114 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-24T05:59:49.114 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-24T05:59:49.114 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-24T05:59:49.115 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-24T05:59:49.115 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-24T05:59:49.115 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-24T05:59:49.115 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-24T05:59:49.115 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-24T05:59:49.115 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-24T05:59:49.115 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-24T05:59:49.115 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-24T05:59:49.115 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-24T05:59:49.115 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-24T05:59:49.115 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-24T05:59:49.115 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-24T05:59:49.116 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-24T05:59:49.116 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-24T05:59:49.116 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-24T05:59:49.116 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-24T05:59:49.116 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-24T05:59:49.116 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-24T05:59:49.117 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-24T05:59:49.117 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-24T05:59:49.117 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-24T05:59:49.117 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-24T05:59:49.117 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-24T05:59:49.117 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-24T05:59:49.117 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-24T05:59:49.117 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-24T05:59:49.117 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-24T05:59:49.117 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-24T05:59:49.117 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-24T05:59:49.117 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-24T05:59:49.118 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-24T05:59:49.118 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-24T05:59:49.118 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-24T05:59:49.118 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-24T05:59:49.118 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-24T05:59:49.118 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-24T05:59:49.118 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-24T05:59:49.118 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-24T05:59:49.118 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-24T05:59:49.118 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-24T05:59:49.118 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-24T05:59:49.119 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-24T05:59:49.119 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-24T05:59:49.119 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-24T05:59:49.119 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-24T05:59:49.119 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-24T05:59:49.119 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-24T05:59:49.119 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-24T05:59:49.119 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-24T05:59:49.119 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-24T05:59:49.119 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-24T05:59:49.119 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-24T05:59:49.119 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-24T05:59:49.120 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-24T05:59:49.120 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-24T05:59:49.120 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-24T05:59:49.120 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-24T05:59:49.120 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-24T05:59:49.120 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-24T05:59:49.120 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-24T05:59:49.120 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-24T05:59:49.120 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-24T05:59:49.120 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-24T05:59:49.120 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-24T05:59:49.120 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-24T05:59:49.121 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-24T05:59:49.121 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-24T05:59:49.121 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-24T05:59:49.121 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-24T05:59:49.121 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-24T05:59:49.121 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-24T05:59:49.121 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-24T05:59:49.121 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-24T05:59:49.121 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-24T05:59:49.121 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-24T05:59:49.121 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-24T05:59:49.121 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-24T05:59:49.122 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-24T05:59:49.122 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-24T05:59:49.122 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-24T05:59:49.122 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-24T05:59:49.122 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-24T05:59:49.122 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-24T05:59:49.122 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-24T05:59:49.122 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-24T05:59:49.122 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-24T05:59:49.122 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-24T05:59:49.122 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-24T05:59:49.123 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-24T05:59:49.123 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-24T05:59:49.123 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-24T05:59:49.123 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-24T05:59:49.123 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-24T05:59:49.123 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-24T05:59:49.123 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-24T05:59:49.123 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-24T05:59:49.123 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-24T05:59:49.123 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-24T05:59:49.123 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-24T05:59:49.123 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-24T05:59:49.124 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-24T05:59:49.124 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-24T05:59:49.124 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-24T05:59:49.124 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-24T05:59:49.124 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-24T05:59:49.124 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-24T05:59:49.124 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-24T05:59:49.124 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-24T05:59:49.124 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-24T05:59:49.124 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-24T05:59:49.124 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-24T05:59:49.125 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-24T05:59:49.125 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-24T05:59:49.125 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-24T05:59:49.125 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-24T05:59:49.125 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-24T05:59:49.125 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-24T05:59:49.125 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-24T05:59:49.125 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-24T05:59:49.126 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-24T05:59:49.126 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-24T05:59:49.126 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-24T05:59:49.126 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-24T05:59:49.126 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-24T05:59:49.126 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-24T05:59:49.126 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-24T05:59:49.126 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-24T05:59:49.126 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-24T05:59:49.127 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-24T05:59:49.127 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-24T05:59:49.127 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-24T05:59:49.127 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-24T05:59:49.127 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-24T05:59:49.127 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-24T05:59:49.127 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-24T05:59:49.127 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-24T05:59:49.127 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-24T05:59:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-24T05:59:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-24T05:59:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-24T05:59:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-24T05:59:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-24T05:59:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-24T05:59:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-24T05:59:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-24T05:59:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-24T05:59:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-24T05:59:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-24T05:59:49.128 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-24T05:59:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-24T05:59:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-24T05:59:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-24T05:59:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-24T05:59:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-24T05:59:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-24T05:59:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-24T05:59:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-24T05:59:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-24T05:59:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-24T05:59:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-24T05:59:49.129 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-24T05:59:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-24T05:59:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-24T05:59:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-24T05:59:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-24T05:59:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-24T05:59:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-24T05:59:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-24T05:59:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-24T05:59:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-24T05:59:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-24T05:59:49.130 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-24T05:59:49.131 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-24T05:59:49.131 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-24T05:59:49.131 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-24T05:59:49.131 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-24T05:59:49.131 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-24T05:59:49.131 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-24T05:59:49.131 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-24T05:59:49.131 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-24T05:59:49.131 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-24T05:59:49.131 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-24T05:59:49.131 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-24T05:59:49.131 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-24T05:59:49.132 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-24T05:59:49.132 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-24T05:59:49.132 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-24T05:59:49.132 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-24T05:59:49.132 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-24T05:59:49.132 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-24T05:59:49.132 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-24T05:59:49.132 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-24T05:59:49.132 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-24T05:59:49.132 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-24T05:59:49.132 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-24T05:59:49.132 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-24T05:59:49.133 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-24T05:59:49.133 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-24T05:59:49.133 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-24T05:59:49.133 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-24T05:59:49.133 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-24T05:59:49.133 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-24T05:59:49.133 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-24T05:59:49.133 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-24T05:59:49.133 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-24T05:59:49.133 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-24T05:59:49.133 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-24T05:59:49.134 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-24T05:59:49.134 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-24T05:59:49.134 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-24T05:59:49.134 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-24T05:59:49.134 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-24T05:59:49.134 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-24T05:59:49.134 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-24T05:59:49.134 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-24T05:59:49.134 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-24T05:59:49.134 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-24T05:59:49.134 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-24T05:59:49.135 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-24T05:59:49.135 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-24T05:59:49.135 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-24T05:59:49.135 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-24T05:59:49.135 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-24T05:59:49.135 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-24T05:59:49.135 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-24T05:59:49.135 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-24T05:59:49.135 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-24T05:59:49.135 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-24T05:59:49.135 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-24T05:59:49.135 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-24T05:59:49.136 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-24T05:59:49.136 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-24T05:59:49.136 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-24T05:59:49.136 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-24T05:59:49.136 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-24T05:59:49.136 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-24T05:59:49.136 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-24T05:59:49.136 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-24T05:59:49.136 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-24T05:59:49.136 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-24T05:59:49.136 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-24T05:59:49.136 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-24T05:59:49.137 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-24T05:59:49.137 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-24T05:59:49.138 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-24T05:59:49.138 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-24T05:59:49.138 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-24T05:59:49.139 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-24T05:59:49.139 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-24T05:59:49.139 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-24T05:59:49.139 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-24T05:59:49.139 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-24T05:59:49.140 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-24T05:59:49.140 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-24T05:59:49.140 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-24T05:59:49.140 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-24T05:59:49.140 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-24T05:59:49.141 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-24T05:59:49.141 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-24T05:59:49.141 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-24T05:59:49.141 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-24T05:59:49.141 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-24T05:59:49.141 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-24T05:59:49.141 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-24T05:59:49.141 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-24T05:59:49.142 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-24T05:59:49.142 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-24T05:59:49.142 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-24T05:59:49.142 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-24T05:59:49.142 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-24T05:59:49.142 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-24T05:59:49.142 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-24T05:59:49.142 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-24T05:59:49.142 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-24T05:59:49.142 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-24T05:59:49.142 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-24T05:59:49.142 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-24T05:59:49.143 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-24T05:59:49.143 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-24T05:59:49.143 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-24T05:59:49.143 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-24T05:59:49.143 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-24T05:59:49.143 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-24T05:59:49.143 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-24T05:59:49.143 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-24T05:59:49.143 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-24T05:59:49.143 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-24T05:59:49.143 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-24T05:59:49.143 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-24T05:59:49.144 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-24T05:59:49.144 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-24T05:59:49.144 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-24T05:59:49.144 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-24T05:59:49.144 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-24T05:59:49.144 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-24T05:59:49.144 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-24T05:59:49.145 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-24T05:59:49.145 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-24T05:59:49.145 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-24T05:59:49.145 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-24T05:59:49.145 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-24T05:59:49.145 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-24T05:59:49.145 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-24T05:59:49.145 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-24T05:59:49.145 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-24T05:59:49.146 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-24T05:59:49.146 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-24T05:59:49.146 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-24T05:59:49.146 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-24T05:59:49.146 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-24T05:59:49.146 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-24T05:59:49.148 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-24T05:59:49.148 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-24T05:59:49.148 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-24T05:59:49.148 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-24T05:59:49.148 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-24T05:59:49.148 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-24T05:59:49.149 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-24T05:59:49.149 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-24T05:59:49.149 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-24T05:59:49.149 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-24T05:59:49.149 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-24T05:59:49.149 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-24T05:59:49.149 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-24T05:59:49.149 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-24T05:59:49.149 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-24T05:59:49.149 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-24T05:59:49.149 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-24T05:59:49.149 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-24T05:59:49.150 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-24T05:59:49.150 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-24T05:59:49.150 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-24T05:59:49.150 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-24T05:59:49.150 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-24T05:59:49.150 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-24T05:59:49.150 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-24T05:59:49.150 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-24T05:59:49.150 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-24T05:59:49.150 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-24T05:59:49.151 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-24T05:59:49.151 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-24T05:59:49.151 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-24T05:59:49.151 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-24T05:59:49.151 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-24T05:59:49.151 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-24T05:59:49.151 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-24T05:59:49.151 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-24T05:59:49.151 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-24T05:59:49.151 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-24T05:59:49.152 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-24T05:59:49.152 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-24T05:59:49.152 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-24T05:59:49.152 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-24T05:59:49.152 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-24T05:59:49.152 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-24T05:59:49.152 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-24T05:59:49.152 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-24T05:59:49.152 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-24T05:59:49.152 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-24T05:59:49.152 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-24T05:59:49.152 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-24T05:59:49.152 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-24T05:59:49.153 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-24T05:59:49.153 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-24T05:59:49.153 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-24T05:59:49.153 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-24T05:59:49.153 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-24T05:59:49.153 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-24T05:59:49.153 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-24T05:59:49.153 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-24T05:59:49.153 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-24T05:59:49.153 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-24T05:59:49.153 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-24T05:59:49.153 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-24T05:59:49.154 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-24T05:59:49.154 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-24T05:59:49.154 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-24T05:59:49.154 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-24T05:59:49.154 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-24T05:59:49.154 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-24T05:59:49.154 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-24T05:59:49.154 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-24T05:59:49.154 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-24T05:59:49.154 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-24T05:59:49.154 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-24T05:59:49.155 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-24T05:59:49.155 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-24T05:59:49.155 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-24T05:59:49.155 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-24T05:59:49.155 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-24T05:59:49.155 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-24T05:59:49.155 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-24T05:59:49.155 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-24T05:59:49.156 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-24T05:59:49.156 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-24T05:59:49.156 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-24T05:59:49.156 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-24T05:59:49.156 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-24T05:59:49.156 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-24T05:59:49.156 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-24T05:59:49.156 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-24T05:59:49.156 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-24T05:59:49.156 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-24T05:59:49.156 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-24T05:59:49.157 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-24T05:59:49.157 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-24T05:59:49.157 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-24T05:59:49.157 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-24T05:59:49.157 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-24T05:59:49.157 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-24T05:59:49.157 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-24T05:59:49.157 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-24T05:59:49.157 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-24T05:59:49.157 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-24T05:59:49.157 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-24T05:59:49.157 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-24T05:59:49.158 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-24T05:59:49.158 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-24T05:59:49.158 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-24T05:59:49.158 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-24T05:59:49.158 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-24T05:59:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-24T05:59:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-24T05:59:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-24T05:59:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-24T05:59:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-24T05:59:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-24T05:59:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-24T05:59:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-24T05:59:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-24T05:59:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-24T05:59:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-24T05:59:49.159 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-24T05:59:49.160 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-24T05:59:49.160 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-24T05:59:49.160 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-24T05:59:49.160 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-24T05:59:49.160 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-24T05:59:49.160 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-24T05:59:49.160 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-24T05:59:49.160 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-24T05:59:49.160 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-24T05:59:49.160 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-24T05:59:49.160 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-24T05:59:49.160 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-24T05:59:49.161 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-24T05:59:49.161 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-24T05:59:49.161 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-24T05:59:49.161 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-24T05:59:49.161 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-24T05:59:49.161 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-24T05:59:49.161 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-24T05:59:49.161 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-24T05:59:49.161 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-24T05:59:49.161 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-24T05:59:49.161 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-24T05:59:49.162 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-24T05:59:49.162 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-24T05:59:49.162 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-24T05:59:49.162 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-24T05:59:49.162 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-24T05:59:49.162 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-24T05:59:49.162 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-24T05:59:49.162 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-24T05:59:49.162 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-24T05:59:49.162 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-24T05:59:49.162 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-24T05:59:49.162 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-24T05:59:49.163 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-24T05:59:49.163 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-24T05:59:49.163 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-24T05:59:49.163 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-24T05:59:49.163 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-24T05:59:49.163 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-24T05:59:49.163 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-24T05:59:49.163 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-24T05:59:49.163 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-24T05:59:49.163 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-24T05:59:49.163 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-24T05:59:49.164 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-24T05:59:49.164 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-24T05:59:49.164 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-24T05:59:49.164 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-24T05:59:49.164 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-24T05:59:49.164 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-24T05:59:49.164 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-24T05:59:49.164 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-24T05:59:49.164 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-24T05:59:49.164 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-24T05:59:49.164 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-24T05:59:49.164 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-24T05:59:49.165 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-24T05:59:49.165 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-24T05:59:49.165 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-24T05:59:49.165 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-24T05:59:49.165 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-24T05:59:49.165 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-24T05:59:49.165 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-24T05:59:49.166 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-24T05:59:49.166 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-24T05:59:49.166 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-24T05:59:49.166 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-24T05:59:49.166 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-24T05:59:49.166 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-24T05:59:49.166 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-24T05:59:49.166 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-24T05:59:49.166 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-24T05:59:49.166 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-24T05:59:49.166 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-24T05:59:49.167 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-24T05:59:49.167 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-24T05:59:49.167 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-24T05:59:49.167 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-24T05:59:49.167 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-24T05:59:49.167 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-24T05:59:49.167 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-24T05:59:49.167 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-24T05:59:49.167 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-24T05:59:49.167 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-24T05:59:49.167 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-24T05:59:49.167 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-24T05:59:49.167 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-24T05:59:49.168 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-24T05:59:49.168 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-24T05:59:49.168 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-24T05:59:49.168 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-24T05:59:49.168 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-24T05:59:49.168 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-24T05:59:49.168 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-24T05:59:49.168 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-24T05:59:49.168 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-24T05:59:49.168 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-24T05:59:49.169 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-24T05:59:49.169 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-24T05:59:49.169 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-24T05:59:49.169 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-24T05:59:49.169 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-24T05:59:49.169 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-24T05:59:49.170 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-24T05:59:49.170 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-24T05:59:49.170 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-24T05:59:49.170 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-24T05:59:49.170 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-24T05:59:49.170 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-24T05:59:49.170 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-24T05:59:49.170 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-24T05:59:49.170 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-24T05:59:49.170 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-24T05:59:49.170 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-24T05:59:49.170 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-24T05:59:49.170 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-24T05:59:49.171 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-24T05:59:49.171 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-24T05:59:49.171 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-24T05:59:49.171 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-24T05:59:49.171 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-24T05:59:49.171 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-24T05:59:49.171 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-24T05:59:49.171 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-24T05:59:49.171 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-24T05:59:49.171 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-24T05:59:49.171 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-24T05:59:49.171 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-24T05:59:49.172 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-24T05:59:49.172 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-24T05:59:49.172 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-24T05:59:49.172 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-24T05:59:49.172 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-24T05:59:49.172 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-24T05:59:49.172 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-24T05:59:49.172 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-24T05:59:49.172 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-24T05:59:49.172 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-24T05:59:49.172 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-24T05:59:49.172 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-24T05:59:49.173 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-24T05:59:49.173 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-24T05:59:49.173 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-24T05:59:49.173 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-24T05:59:49.173 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-24T05:59:49.173 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-24T05:59:49.173 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-24T05:59:49.173 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-24T05:59:49.173 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-24T05:59:49.173 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-24T05:59:49.173 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-24T05:59:49.173 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-24T05:59:49.174 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-24T05:59:49.174 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-24T05:59:49.174 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-24T05:59:49.174 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-24T05:59:49.174 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-24T05:59:49.174 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-24T05:59:49.174 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-24T05:59:49.174 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-24T05:59:49.174 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-24T05:59:49.174 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-24T05:59:49.174 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-24T05:59:49.175 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-24T05:59:49.175 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-24T05:59:49.175 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-24T05:59:49.175 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-24T05:59:49.175 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-24T05:59:49.175 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-24T05:59:49.175 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-24T05:59:49.175 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-24T05:59:49.176 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-24T05:59:49.176 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-24T05:59:49.176 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-24T05:59:49.176 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-24T05:59:49.176 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-24T05:59:49.176 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-24T05:59:49.176 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-24T05:59:49.176 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-24T05:59:49.176 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-24T05:59:49.177 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-24T05:59:49.177 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-24T05:59:49.177 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-24T05:59:49.177 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-24T05:59:49.177 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-24T05:59:49.177 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-24T05:59:49.177 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-24T05:59:49.177 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-24T05:59:49.177 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-24T05:59:49.177 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-24T05:59:49.177 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-24T05:59:49.177 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-24T05:59:49.178 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-24T05:59:49.178 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-24T05:59:49.178 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-24T05:59:49.178 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-24T05:59:49.178 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-24T05:59:49.178 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-24T05:59:49.178 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-24T05:59:49.178 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-24T05:59:49.178 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-24T05:59:49.178 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-24T05:59:49.178 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-24T05:59:49.178 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-24T05:59:49.179 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-24T05:59:49.179 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-24T05:59:49.179 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-24T05:59:49.179 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-24T05:59:49.179 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-24T05:59:49.179 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-24T05:59:49.180 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-24T05:59:49.180 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-24T05:59:49.180 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-24T05:59:49.180 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-24T05:59:49.180 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-24T05:59:49.180 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-24T05:59:49.180 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-24T05:59:49.180 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-24T05:59:49.180 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-24T05:59:49.180 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-24T05:59:49.180 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-24T05:59:49.181 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-24T05:59:49.181 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-24T05:59:49.181 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-24T05:59:49.181 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-24T05:59:49.181 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-24T05:59:49.181 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-24T05:59:49.181 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-24T05:59:49.181 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-24T05:59:49.181 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-24T05:59:49.181 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-24T05:59:49.181 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-24T05:59:49.181 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-24T05:59:49.182 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-24T05:59:49.182 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-24T05:59:49.182 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-24T05:59:49.182 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-24T05:59:49.182 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-24T05:59:49.182 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-24T05:59:49.182 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-24T05:59:49.182 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-24T05:59:49.182 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-24T05:59:49.182 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-24T05:59:49.182 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-24T05:59:49.182 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-24T05:59:49.183 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-24T05:59:49.183 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-24T05:59:49.183 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-24T05:59:49.183 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-24T05:59:49.183 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-24T05:59:49.183 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-24T05:59:49.183 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-24T05:59:49.183 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-24T05:59:49.183 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-24T05:59:49.183 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-24T05:59:49.183 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-24T05:59:49.183 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-24T05:59:49.184 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-24T05:59:49.184 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-24T05:59:49.184 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-24T05:59:49.184 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-24T05:59:49.184 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-24T05:59:49.184 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-24T05:59:49.184 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-24T05:59:49.184 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-24T05:59:49.184 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-24T05:59:49.184 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-24T05:59:49.184 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-24T05:59:49.184 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-24T05:59:49.185 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-24T05:59:49.185 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-24T05:59:49.185 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-24T05:59:49.185 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-24T05:59:49.185 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-24T05:59:49.185 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-24T05:59:49.185 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-24T05:59:49.185 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-24T05:59:49.186 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-24T05:59:49.186 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-24T05:59:49.186 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-24T05:59:49.186 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-24T05:59:49.186 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-24T05:59:49.186 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-24T05:59:49.186 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-24T05:59:49.186 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-24T05:59:49.186 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-24T05:59:49.186 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-24T05:59:49.186 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-24T05:59:49.186 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-24T05:59:49.187 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-24T05:59:49.187 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-24T05:59:49.187 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-24T05:59:49.187 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-24T05:59:49.187 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-24T05:59:49.187 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-24T05:59:49.187 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-24T05:59:49.187 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-24T05:59:49.187 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-24T05:59:49.187 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-24T05:59:49.187 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-24T05:59:49.188 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-24T05:59:49.188 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-24T05:59:49.188 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-24T05:59:49.188 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-24T05:59:49.188 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-24T05:59:49.188 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-24T05:59:49.188 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-24T05:59:49.188 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-24T05:59:49.188 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-24T05:59:49.188 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-24T05:59:49.189 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-24T05:59:49.189 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-24T05:59:49.189 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-24T05:59:49.189 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-24T05:59:49.189 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-24T05:59:49.189 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-24T05:59:49.190 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-24T05:59:49.190 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-24T05:59:49.190 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-24T05:59:49.191 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-24T05:59:49.191 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-24T05:59:49.191 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-24T05:59:49.191 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-24T05:59:49.191 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-24T05:59:49.192 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-24T05:59:49.192 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-24T05:59:49.192 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-24T05:59:49.192 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-24T05:59:49.192 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-24T05:59:49.192 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-24T05:59:49.193 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-24T05:59:49.193 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-24T05:59:49.193 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-24T05:59:49.193 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-24T05:59:49.193 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-24T05:59:49.193 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-24T05:59:49.194 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-24T05:59:49.194 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-24T05:59:49.194 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-24T05:59:49.194 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-24T05:59:49.194 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-24T05:59:49.194 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-24T05:59:49.195 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-24T05:59:49.195 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-24T05:59:49.195 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-24T05:59:49.195 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-24T05:59:49.196 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-24T05:59:49.196 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-24T05:59:49.196 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-24T05:59:49.196 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-24T05:59:49.196 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-24T05:59:49.196 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-24T05:59:49.196 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-24T05:59:49.197 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-24T05:59:49.197 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-24T05:59:49.197 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-24T05:59:49.197 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-24T05:59:49.197 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-24T05:59:49.197 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-24T05:59:49.198 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-24T05:59:49.198 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-24T05:59:49.198 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-24T05:59:49.198 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-24T05:59:49.198 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-24T05:59:49.198 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-24T05:59:49.199 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-24T05:59:49.199 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-24T05:59:49.199 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-24T05:59:49.199 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-24T05:59:49.199 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-24T05:59:49.200 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-24T05:59:49.200 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-24T05:59:49.200 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-24T05:59:49.201 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-24T05:59:49.201 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-24T05:59:49.202 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-24T05:59:49.202 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-24T05:59:49.202 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-24T05:59:49.202 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-24T05:59:49.202 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-24T05:59:49.203 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-24T05:59:49.203 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-24T05:59:49.203 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-24T05:59:49.203 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-24T05:59:49.203 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-24T05:59:49.204 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-24T05:59:49.204 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-24T05:59:49.204 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-24T05:59:49.204 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-24T05:59:49.204 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-24T05:59:49.205 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-24T05:59:49.205 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-24T05:59:49.205 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-24T05:59:49.205 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-24T05:59:49.205 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-24T05:59:49.205 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-24T05:59:49.206 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-24T05:59:49.206 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-24T05:59:49.206 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-24T05:59:49.206 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-24T05:59:49.206 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-24T05:59:49.207 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-24T05:59:49.207 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-24T05:59:49.207 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-24T05:59:49.207 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-24T05:59:49.207 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-24T05:59:49.208 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-24T05:59:49.208 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-24T05:59:49.208 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-24T05:59:49.208 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-24T05:59:49.208 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-24T05:59:49.209 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-24T05:59:49.209 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-24T05:59:49.209 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-24T05:59:49.209 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-24T05:59:49.209 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-24T05:59:49.210 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-24T05:59:49.210 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-24T05:59:49.210 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-24T05:59:49.210 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-24T05:59:49.210 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-24T05:59:49.211 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-24T05:59:49.211 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-24T05:59:49.211 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-24T05:59:49.211 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-24T05:59:49.214 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-24T05:59:49.214 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-24T05:59:49.215 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-24T05:59:49.215 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-24T05:59:49.215 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-24T05:59:49.215 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-24T05:59:49.215 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-24T05:59:49.215 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-24T05:59:49.216 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-24T05:59:49.216 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-24T05:59:49.216 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-24T05:59:49.216 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-24T05:59:49.216 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-24T05:59:49.217 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-24T05:59:49.217 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-24T05:59:49.217 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-24T05:59:49.217 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-24T05:59:49.217 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-24T05:59:49.218 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-24T05:59:49.218 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-24T05:59:49.218 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-24T05:59:49.218 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-24T05:59:49.218 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-24T05:59:49.219 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-24T05:59:49.219 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-24T05:59:49.219 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-24T05:59:49.219 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-24T05:59:49.219 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-24T05:59:49.220 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-24T05:59:49.220 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-24T05:59:49.220 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-24T05:59:49.220 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-24T05:59:49.220 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-24T05:59:49.221 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-24T05:59:49.221 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-24T05:59:49.221 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-24T05:59:49.221 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-24T05:59:49.221 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-24T05:59:49.221 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-24T05:59:49.222 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-24T05:59:49.222 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-24T05:59:49.222 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-24T05:59:49.222 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-24T05:59:49.222 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-24T05:59:49.222 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-24T05:59:49.223 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-24T05:59:49.223 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-24T05:59:49.223 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-24T05:59:49.223 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-24T05:59:49.223 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-24T05:59:49.224 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-24T05:59:49.224 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-24T05:59:49.224 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-24T05:59:49.225 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-24T05:59:49.225 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-24T05:59:49.225 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-24T05:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-24T05:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-24T05:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-24T05:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-24T05:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-24T05:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-24T05:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-24T05:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-24T05:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-24T05:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-24T05:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-24T05:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-24T05:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-24T05:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-24T05:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-24T05:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-24T05:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-24T05:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-24T05:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-24T05:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-24T05:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-24T05:59:49.230 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-24T05:59:49.230 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-24T05:59:49.230 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-24T05:59:49.230 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-24T05:59:49.230 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-24T05:59:49.231 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-24T05:59:49.231 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-24T05:59:49.231 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-24T05:59:49.231 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-24T05:59:49.231 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-24T05:59:49.232 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-24T05:59:49.232 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-24T05:59:49.232 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-24T05:59:49.232 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-24T05:59:49.232 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-24T05:59:49.233 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-24T05:59:49.233 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-24T05:59:49.233 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-24T05:59:49.233 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-24T05:59:49.233 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-24T05:59:49.234 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-24T05:59:49.234 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-24T05:59:49.234 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-24T05:59:49.234 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-24T05:59:49.234 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-24T05:59:49.235 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-24T05:59:49.235 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-24T05:59:49.236 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-24T05:59:49.237 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-24T05:59:49.237 DEBUG:teuthology.orchestra.run.smithi005:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-24T05:59:49.265 INFO:teuthology.orchestra.run.smithi005.stdout:postinstall scriptlet (using /bin/sh): 2023-11-24T05:59:49.265 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-24T05:59:49.265 INFO:teuthology.orchestra.run.smithi005.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:59:49.265 INFO:teuthology.orchestra.run.smithi005.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:59:49.265 INFO:teuthology.orchestra.run.smithi005.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T05:59:49.265 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:59:49.265 INFO:teuthology.orchestra.run.smithi005.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:59:49.265 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2023-11-24T05:59:49.265 INFO:teuthology.orchestra.run.smithi005.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-24T05:59:49.266 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-24T05:59:49.266 INFO:teuthology.orchestra.run.smithi005.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-24T05:59:49.266 INFO:teuthology.orchestra.run.smithi005.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-24T05:59:49.266 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-24T05:59:49.266 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2023-11-24T05:59:49.266 INFO:teuthology.orchestra.run.smithi005.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-24T05:59:49.266 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-24T05:59:49.266 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-24T05:59:49.266 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2023-11-24T05:59:49.267 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2023-11-24T05:59:49.352 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2023-11-24T05:59:49.352 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2023-11-24T05:59:49.352 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.06 2023-11-24T05:59:49.352 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 68.el9 2023-11-24T05:59:49.352 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2023-11-24T05:59:49.353 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-24T05:59:49.353 INFO:teuthology.orchestra.run.smithi005.stdout:Group : Unspecified 2023-11-24T05:59:49.353 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 8264254 2023-11-24T05:59:49.353 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2023-11-24T05:59:49.353 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-24T05:59:49.353 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-24T05:59:49.353 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-24T05:59:49.353 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-24T05:59:49.353 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : builder@centos.org 2023-11-24T05:59:49.353 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : CentOS 2023-11-24T05:59:49.353 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-24T05:59:49.353 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2023-11-24T05:59:49.354 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2023-11-24T05:59:49.354 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T05:59:49.354 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-24T05:59:49.354 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-24T05:59:49.354 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-24T05:59:49.354 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2023-11-24T05:59:49.354 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T05:59:49.354 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2023-11-24T05:59:49.355 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-24T05:59:49.355 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 6.6.0-g8247bbb5b470 2023-11-24T05:59:49.355 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-24T05:59:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2023-11-24T05:59:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-24T05:59:51.400 INFO:teuthology.orchestra.run.smithi005.stderr:done 2023-11-24T05:59:51.402 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2023-11-24T05:59:51.417 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.1idpmEq0u1 2023-11-24T05:59:51.417 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1idpmEq0u1 2023-11-24T05:59:51.499 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.1idpmEq0u1 2023-11-24T05:59:51.645 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.1idpmEq0u1 is 6KB 2023-11-24T05:59:51.696 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.1idpmEq0u1 2023-11-24T05:59:51.711 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2023-11-24T05:59:51.779 INFO:teuthology.orchestra.run.smithi005.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-24T05:59:51.779 INFO:teuthology.orchestra.run.smithi005.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-24T05:59:51.779 INFO:teuthology.orchestra.run.smithi005.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-24T05:59:51.780 INFO:teuthology.orchestra.run.smithi005.stdout:15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470.conf 2023-11-24T05:59:51.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470 2023-11-24T05:59:51.938 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2023-11-24T05:59:52.941 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi006.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-24T05:59:52.941 DEBUG:teuthology.orchestra.run.smithi006:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-24T05:59:53.054 INFO:teuthology.orchestra.run.smithi006.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T05:59:53.054 INFO:teuthology.orchestra.run.smithi006.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-24T05:59:53.055 INFO:teuthology.orchestra.run.smithi006.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-24T05:59:53.055 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-24T05:59:53.055 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-24T05:59:53.055 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-24T05:59:53.055 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-24T05:59:53.056 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-24T05:59:53.056 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-24T05:59:53.056 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-24T05:59:53.056 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-24T05:59:53.057 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-24T05:59:53.057 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-24T05:59:53.057 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-24T05:59:53.057 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-24T05:59:53.057 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-24T05:59:53.058 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-24T05:59:53.058 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-24T05:59:53.058 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-24T05:59:53.058 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-24T05:59:53.058 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-24T05:59:53.059 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-24T05:59:53.059 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-24T05:59:53.059 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-24T05:59:53.059 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-24T05:59:53.059 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-24T05:59:53.059 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-24T05:59:53.060 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-24T05:59:53.060 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-24T05:59:53.060 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-24T05:59:53.060 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-24T05:59:53.060 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-24T05:59:53.060 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-24T05:59:53.061 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-24T05:59:53.061 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-24T05:59:53.061 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-24T05:59:53.061 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-24T05:59:53.061 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-24T05:59:53.061 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-24T05:59:53.062 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-24T05:59:53.062 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-24T05:59:53.062 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-24T05:59:53.062 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-24T05:59:53.062 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-24T05:59:53.063 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-24T05:59:53.063 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-24T05:59:53.063 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-24T05:59:53.063 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-24T05:59:53.063 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-24T05:59:53.064 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-24T05:59:53.064 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-24T05:59:53.064 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-24T05:59:53.064 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-24T05:59:53.064 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-24T05:59:53.065 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-24T05:59:53.065 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-24T05:59:53.065 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-24T05:59:53.065 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-24T05:59:53.065 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-24T05:59:53.066 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-24T05:59:53.066 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-24T05:59:53.066 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-24T05:59:53.066 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-24T05:59:53.066 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-24T05:59:53.066 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-24T05:59:53.067 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-24T05:59:53.067 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-24T05:59:53.067 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-24T05:59:53.068 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-24T05:59:53.068 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-24T05:59:53.068 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-24T05:59:53.068 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-24T05:59:53.068 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-24T05:59:53.069 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-24T05:59:53.069 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-24T05:59:53.069 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-24T05:59:53.069 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-24T05:59:53.069 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-24T05:59:53.069 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-24T05:59:53.070 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-24T05:59:53.070 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-24T05:59:53.070 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-24T05:59:53.070 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-24T05:59:53.071 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-24T05:59:53.071 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-24T05:59:53.071 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-24T05:59:53.074 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-24T05:59:53.074 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-24T05:59:53.075 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-24T05:59:53.075 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-24T05:59:53.075 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-24T05:59:53.076 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-24T05:59:53.076 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-24T05:59:53.076 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-24T05:59:53.077 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-24T05:59:53.077 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-24T05:59:53.077 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-24T05:59:53.077 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-24T05:59:53.078 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-24T05:59:53.078 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-24T05:59:53.078 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-24T05:59:53.078 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-24T05:59:53.078 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-24T05:59:53.079 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-24T05:59:53.079 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-24T05:59:53.079 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-24T05:59:53.079 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-24T05:59:53.080 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-24T05:59:53.081 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-24T05:59:53.081 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-24T05:59:53.081 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-24T05:59:53.081 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-24T05:59:53.081 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-24T05:59:53.081 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-24T05:59:53.082 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-24T05:59:53.083 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-24T05:59:53.083 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-24T05:59:53.083 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-24T05:59:53.083 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-24T05:59:53.084 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-24T05:59:53.084 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-24T05:59:53.084 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-24T05:59:53.085 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-24T05:59:53.085 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-24T05:59:53.086 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-24T05:59:53.086 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-24T05:59:53.086 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-24T05:59:53.086 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-24T05:59:53.088 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-24T05:59:53.089 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-24T05:59:53.089 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-24T05:59:53.089 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-24T05:59:53.089 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-24T05:59:53.089 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-24T05:59:53.090 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-24T05:59:53.090 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-24T05:59:53.090 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-24T05:59:53.091 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-24T05:59:53.091 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-24T05:59:53.091 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-24T05:59:53.092 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-24T05:59:53.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-24T05:59:53.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-24T05:59:53.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-24T05:59:53.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-24T05:59:53.093 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-24T05:59:53.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-24T05:59:53.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-24T05:59:53.094 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-24T05:59:53.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-24T05:59:53.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-24T05:59:53.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-24T05:59:53.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-24T05:59:53.099 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-24T05:59:53.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-24T05:59:53.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-24T05:59:53.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-24T05:59:53.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-24T05:59:53.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-24T05:59:53.100 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-24T05:59:53.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-24T05:59:53.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-24T05:59:53.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-24T05:59:53.101 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-24T05:59:53.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-24T05:59:53.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-24T05:59:53.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-24T05:59:53.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-24T05:59:53.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-24T05:59:53.102 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-24T05:59:53.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-24T05:59:53.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-24T05:59:53.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-24T05:59:53.103 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-24T05:59:53.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-24T05:59:53.111 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-24T05:59:53.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-24T05:59:53.112 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-24T05:59:53.115 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-24T05:59:53.115 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-24T05:59:53.115 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-24T05:59:53.116 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-24T05:59:53.116 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-24T05:59:53.116 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-24T05:59:53.116 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-24T05:59:53.116 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-24T05:59:53.117 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-24T05:59:53.117 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-24T05:59:53.117 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-24T05:59:53.117 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-24T05:59:53.117 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-24T05:59:53.117 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-24T05:59:53.118 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-24T05:59:53.118 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-24T05:59:53.118 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-24T05:59:53.118 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-24T05:59:53.118 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-24T05:59:53.119 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-24T05:59:53.119 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-24T05:59:53.119 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-24T05:59:53.119 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-24T05:59:53.119 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-24T05:59:53.119 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-24T05:59:53.120 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-24T05:59:53.120 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-24T05:59:53.120 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-24T05:59:53.121 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-24T05:59:53.121 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-24T05:59:53.121 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-24T05:59:53.121 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-24T05:59:53.121 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-24T05:59:53.121 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-24T05:59:53.122 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-24T05:59:53.122 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-24T05:59:53.122 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-24T05:59:53.122 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-24T05:59:53.122 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-24T05:59:53.123 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-24T05:59:53.123 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-24T05:59:53.123 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-24T05:59:53.123 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-24T05:59:53.123 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-24T05:59:53.123 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-24T05:59:53.124 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-24T05:59:53.124 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-24T05:59:53.124 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-24T05:59:53.124 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-24T05:59:53.124 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-24T05:59:53.124 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-24T05:59:53.125 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-24T05:59:53.125 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-24T05:59:53.125 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-24T05:59:53.125 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-24T05:59:53.125 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-24T05:59:53.126 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-24T05:59:53.126 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-24T05:59:53.126 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-24T05:59:53.126 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-24T05:59:53.126 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-24T05:59:53.127 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-24T05:59:53.127 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-24T05:59:53.127 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-24T05:59:53.127 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-24T05:59:53.127 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-24T05:59:53.127 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-24T05:59:53.128 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-24T05:59:53.128 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-24T05:59:53.128 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-24T05:59:53.128 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-24T05:59:53.128 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-24T05:59:53.128 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-24T05:59:53.129 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-24T05:59:53.129 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-24T05:59:53.129 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-24T05:59:53.129 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-24T05:59:53.129 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-24T05:59:53.130 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-24T05:59:53.130 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-24T05:59:53.130 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-24T05:59:53.131 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-24T05:59:53.131 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-24T05:59:53.131 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-24T05:59:53.131 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-24T05:59:53.132 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-24T05:59:53.132 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-24T05:59:53.132 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-24T05:59:53.132 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-24T05:59:53.132 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-24T05:59:53.133 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-24T05:59:53.133 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-24T05:59:53.133 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-24T05:59:53.133 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-24T05:59:53.133 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-24T05:59:53.133 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-24T05:59:53.134 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-24T05:59:53.134 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-24T05:59:53.134 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-24T05:59:53.134 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-24T05:59:53.134 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-24T05:59:53.134 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-24T05:59:53.135 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-24T05:59:53.135 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-24T05:59:53.135 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-24T05:59:53.135 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-24T05:59:53.136 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-24T05:59:53.136 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-24T05:59:53.136 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-24T05:59:53.136 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-24T05:59:53.136 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-24T05:59:53.136 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-24T05:59:53.137 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-24T05:59:53.137 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-24T05:59:53.137 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-24T05:59:53.137 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-24T05:59:53.137 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-24T05:59:53.137 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-24T05:59:53.138 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-24T05:59:53.138 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-24T05:59:53.138 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-24T05:59:53.138 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-24T05:59:53.138 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-24T05:59:53.139 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-24T05:59:53.139 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-24T05:59:53.139 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-24T05:59:53.139 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-24T05:59:53.139 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-24T05:59:53.139 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-24T05:59:53.140 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-24T05:59:53.140 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-24T05:59:53.140 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-24T05:59:53.140 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-24T05:59:53.140 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-24T05:59:53.140 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-24T05:59:53.141 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-24T05:59:53.142 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-24T05:59:53.142 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-24T05:59:53.142 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-24T05:59:53.143 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-24T05:59:53.143 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-24T05:59:53.143 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-24T05:59:53.143 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-24T05:59:53.144 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-24T05:59:53.144 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-24T05:59:53.144 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-24T05:59:53.144 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-24T05:59:53.144 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-24T05:59:53.144 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-24T05:59:53.145 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-24T05:59:53.145 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-24T05:59:53.145 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-24T05:59:53.145 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-24T05:59:53.145 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-24T05:59:53.145 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-24T05:59:53.146 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-24T05:59:53.146 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-24T05:59:53.146 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-24T05:59:53.146 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-24T05:59:53.146 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-24T05:59:53.147 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-24T05:59:53.147 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-24T05:59:53.147 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-24T05:59:53.147 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-24T05:59:53.147 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-24T05:59:53.147 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-24T05:59:53.148 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-24T05:59:53.148 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-24T05:59:53.148 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-24T05:59:53.148 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-24T05:59:53.148 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-24T05:59:53.148 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-24T05:59:53.149 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-24T05:59:53.149 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-24T05:59:53.149 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-24T05:59:53.149 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-24T05:59:53.149 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-24T05:59:53.150 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-24T05:59:53.150 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-24T05:59:53.150 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-24T05:59:53.150 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-24T05:59:53.150 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-24T05:59:53.150 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-24T05:59:53.151 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-24T05:59:53.151 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-24T05:59:53.151 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-24T05:59:53.152 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-24T05:59:53.152 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-24T05:59:53.152 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-24T05:59:53.153 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-24T05:59:53.153 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-24T05:59:53.153 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-24T05:59:53.153 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-24T05:59:53.153 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-24T05:59:53.154 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-24T05:59:53.154 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-24T05:59:53.154 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-24T05:59:53.154 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-24T05:59:53.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-24T05:59:53.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-24T05:59:53.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-24T05:59:53.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-24T05:59:53.155 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-24T05:59:53.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-24T05:59:53.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-24T05:59:53.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-24T05:59:53.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-24T05:59:53.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-24T05:59:53.156 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-24T05:59:53.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-24T05:59:53.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-24T05:59:53.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-24T05:59:53.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-24T05:59:53.157 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-24T05:59:53.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-24T05:59:53.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-24T05:59:53.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-24T05:59:53.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-24T05:59:53.158 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-24T05:59:53.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-24T05:59:53.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-24T05:59:53.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-24T05:59:53.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-24T05:59:53.159 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-24T05:59:53.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-24T05:59:53.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-24T05:59:53.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-24T05:59:53.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-24T05:59:53.160 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-24T05:59:53.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-24T05:59:53.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-24T05:59:53.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-24T05:59:53.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-24T05:59:53.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-24T05:59:53.161 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-24T05:59:53.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-24T05:59:53.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-24T05:59:53.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-24T05:59:53.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-24T05:59:53.162 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-24T05:59:53.164 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-24T05:59:53.164 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-24T05:59:53.165 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-24T05:59:53.165 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-24T05:59:53.165 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-24T05:59:53.165 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-24T05:59:53.165 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-24T05:59:53.165 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-24T05:59:53.166 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-24T05:59:53.166 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-24T05:59:53.166 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-24T05:59:53.166 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-24T05:59:53.166 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-24T05:59:53.167 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-24T05:59:53.167 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-24T05:59:53.167 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-24T05:59:53.167 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-24T05:59:53.167 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-24T05:59:53.168 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-24T05:59:53.168 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-24T05:59:53.168 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-24T05:59:53.168 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-24T05:59:53.168 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-24T05:59:53.169 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-24T05:59:53.169 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-24T05:59:53.169 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-24T05:59:53.169 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-24T05:59:53.169 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-24T05:59:53.169 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-24T05:59:53.170 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-24T05:59:53.170 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-24T05:59:53.170 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-24T05:59:53.170 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-24T05:59:53.170 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-24T05:59:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-24T05:59:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-24T05:59:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-24T05:59:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-24T05:59:53.171 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-24T05:59:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-24T05:59:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-24T05:59:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-24T05:59:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-24T05:59:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-24T05:59:53.172 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-24T05:59:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-24T05:59:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-24T05:59:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-24T05:59:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-24T05:59:53.173 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-24T05:59:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-24T05:59:53.174 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-24T05:59:53.175 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-24T05:59:53.175 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-24T05:59:53.175 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-24T05:59:53.175 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-24T05:59:53.175 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-24T05:59:53.176 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-24T05:59:53.176 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-24T05:59:53.176 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-24T05:59:53.176 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-24T05:59:53.176 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-24T05:59:53.177 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-24T05:59:53.177 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-24T05:59:53.177 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-24T05:59:53.177 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-24T05:59:53.177 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-24T05:59:53.177 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-24T05:59:53.178 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-24T05:59:53.178 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-24T05:59:53.178 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-24T05:59:53.178 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-24T05:59:53.178 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-24T05:59:53.179 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-24T05:59:53.179 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-24T05:59:53.179 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-24T05:59:53.179 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-24T05:59:53.179 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-24T05:59:53.179 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-24T05:59:53.180 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-24T05:59:53.180 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-24T05:59:53.180 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-24T05:59:53.180 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-24T05:59:53.180 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-24T05:59:53.181 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-24T05:59:53.181 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-24T05:59:53.181 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-24T05:59:53.181 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-24T05:59:53.181 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-24T05:59:53.181 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-24T05:59:53.182 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-24T05:59:53.182 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-24T05:59:53.182 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-24T05:59:53.182 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-24T05:59:53.182 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-24T05:59:53.183 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-24T05:59:53.183 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-24T05:59:53.183 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-24T05:59:53.183 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-24T05:59:53.183 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-24T05:59:53.183 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-24T05:59:53.184 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-24T05:59:53.184 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-24T05:59:53.184 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-24T05:59:53.184 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-24T05:59:53.184 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-24T05:59:53.184 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-24T05:59:53.185 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-24T05:59:53.186 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-24T05:59:53.186 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-24T05:59:53.186 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-24T05:59:53.186 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-24T05:59:53.186 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-24T05:59:53.187 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-24T05:59:53.187 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-24T05:59:53.187 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-24T05:59:53.187 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-24T05:59:53.187 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-24T05:59:53.187 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-24T05:59:53.188 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-24T05:59:53.188 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-24T05:59:53.188 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-24T05:59:53.188 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-24T05:59:53.188 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-24T05:59:53.189 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-24T05:59:53.189 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-24T05:59:53.189 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-24T05:59:53.189 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-24T05:59:53.189 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-24T05:59:53.189 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-24T05:59:53.190 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-24T05:59:53.190 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-24T05:59:53.190 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-24T05:59:53.190 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-24T05:59:53.190 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-24T05:59:53.191 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-24T05:59:53.191 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-24T05:59:53.191 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-24T05:59:53.191 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-24T05:59:53.191 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-24T05:59:53.192 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-24T05:59:53.192 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-24T05:59:53.192 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-24T05:59:53.192 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-24T05:59:53.192 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-24T05:59:53.193 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-24T05:59:53.193 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-24T05:59:53.193 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-24T05:59:53.193 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-24T05:59:53.193 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-24T05:59:53.194 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-24T05:59:53.194 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-24T05:59:53.194 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-24T05:59:53.194 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-24T05:59:53.194 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-24T05:59:53.194 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-24T05:59:53.195 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-24T05:59:53.195 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-24T05:59:53.195 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-24T05:59:53.196 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-24T05:59:53.196 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-24T05:59:53.196 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-24T05:59:53.196 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-24T05:59:53.196 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-24T05:59:53.197 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-24T05:59:53.197 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-24T05:59:53.197 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-24T05:59:53.197 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-24T05:59:53.197 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-24T05:59:53.197 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-24T05:59:53.198 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-24T05:59:53.198 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-24T05:59:53.198 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-24T05:59:53.198 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-24T05:59:53.198 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-24T05:59:53.198 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-24T05:59:53.199 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-24T05:59:53.199 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-24T05:59:53.199 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-24T05:59:53.199 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-24T05:59:53.199 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-24T05:59:53.200 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-24T05:59:53.200 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-24T05:59:53.200 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-24T05:59:53.200 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-24T05:59:53.200 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-24T05:59:53.200 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-24T05:59:53.201 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-24T05:59:53.201 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-24T05:59:53.201 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-24T05:59:53.201 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-24T05:59:53.201 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-24T05:59:53.201 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-24T05:59:53.202 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-24T05:59:53.202 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-24T05:59:53.202 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-24T05:59:53.202 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-24T05:59:53.202 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-24T05:59:53.203 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-24T05:59:53.203 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-24T05:59:53.203 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-24T05:59:53.203 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-24T05:59:53.203 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-24T05:59:53.203 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-24T05:59:53.204 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-24T05:59:53.204 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-24T05:59:53.204 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-24T05:59:53.204 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-24T05:59:53.204 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-24T05:59:53.205 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-24T05:59:53.205 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-24T05:59:53.205 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-24T05:59:53.205 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-24T05:59:53.205 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-24T05:59:53.205 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-24T05:59:53.206 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-24T05:59:53.206 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-24T05:59:53.206 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-24T05:59:53.207 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-24T05:59:53.207 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-24T05:59:53.207 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-24T05:59:53.207 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-24T05:59:53.207 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-24T05:59:53.208 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-24T05:59:53.208 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-24T05:59:53.208 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-24T05:59:53.208 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-24T05:59:53.208 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-24T05:59:53.208 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-24T05:59:53.209 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-24T05:59:53.209 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-24T05:59:53.209 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-24T05:59:53.209 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-24T05:59:53.209 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-24T05:59:53.210 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-24T05:59:53.210 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-24T05:59:53.210 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-24T05:59:53.210 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-24T05:59:53.210 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-24T05:59:53.211 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-24T05:59:53.211 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-24T05:59:53.211 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-24T05:59:53.211 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-24T05:59:53.211 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-24T05:59:53.211 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-24T05:59:53.212 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-24T05:59:53.212 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-24T05:59:53.212 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-24T05:59:53.212 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-24T05:59:53.212 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-24T05:59:53.212 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-24T05:59:53.213 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-24T05:59:53.213 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-24T05:59:53.213 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-24T05:59:53.213 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-24T05:59:53.213 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-24T05:59:53.214 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-24T05:59:53.214 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-24T05:59:53.214 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-24T05:59:53.214 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-24T05:59:53.214 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-24T05:59:53.214 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-24T05:59:53.215 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-24T05:59:53.215 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-24T05:59:53.215 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-24T05:59:53.215 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-24T05:59:53.215 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-24T05:59:53.216 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-24T05:59:53.216 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-24T05:59:53.216 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-24T05:59:53.216 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-24T05:59:53.217 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-24T05:59:53.217 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-24T05:59:53.217 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-24T05:59:53.217 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-24T05:59:53.217 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-24T05:59:53.218 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-24T05:59:53.218 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-24T05:59:53.218 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-24T05:59:53.218 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-24T05:59:53.218 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-24T05:59:53.218 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-24T05:59:53.219 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-24T05:59:53.219 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-24T05:59:53.219 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-24T05:59:53.219 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-24T05:59:53.219 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-24T05:59:53.220 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-24T05:59:53.220 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-24T05:59:53.220 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-24T05:59:53.220 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-24T05:59:53.220 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-24T05:59:53.220 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-24T05:59:53.221 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-24T05:59:53.221 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-24T05:59:53.221 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-24T05:59:53.221 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-24T05:59:53.221 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-24T05:59:53.221 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-24T05:59:53.222 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-24T05:59:53.222 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-24T05:59:53.222 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-24T05:59:53.222 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-24T05:59:53.222 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-24T05:59:53.223 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-24T05:59:53.223 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-24T05:59:53.223 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-24T05:59:53.223 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-24T05:59:53.223 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-24T05:59:53.223 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-24T05:59:53.224 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-24T05:59:53.224 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-24T05:59:53.224 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-24T05:59:53.224 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-24T05:59:53.224 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-24T05:59:53.224 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-24T05:59:53.225 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-24T05:59:53.225 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-24T05:59:53.225 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-24T05:59:53.225 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-24T05:59:53.225 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-24T05:59:53.226 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-24T05:59:53.226 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-24T05:59:53.226 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-24T05:59:53.226 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-24T05:59:53.226 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-24T05:59:53.227 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-24T05:59:53.227 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-24T05:59:53.227 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-24T05:59:53.228 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-24T05:59:53.228 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-24T05:59:53.228 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-24T05:59:53.228 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-24T05:59:53.228 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-24T05:59:53.228 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-24T05:59:53.229 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-24T05:59:53.229 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-24T05:59:53.229 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-24T05:59:53.229 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-24T05:59:53.229 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-24T05:59:53.230 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-24T05:59:53.230 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-24T05:59:53.230 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-24T05:59:53.230 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-24T05:59:53.230 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-24T05:59:53.230 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-24T05:59:53.231 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-24T05:59:53.231 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-24T05:59:53.231 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-24T05:59:53.231 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-24T05:59:53.231 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-24T05:59:53.232 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-24T05:59:53.232 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-24T05:59:53.232 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-24T05:59:53.232 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-24T05:59:53.232 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-24T05:59:53.232 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-24T05:59:53.233 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-24T05:59:53.233 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-24T05:59:53.233 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-24T05:59:53.233 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-24T05:59:53.233 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-24T05:59:53.233 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-24T05:59:53.234 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-24T05:59:53.234 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-24T05:59:53.234 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-24T05:59:53.234 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-24T05:59:53.234 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-24T05:59:53.235 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-24T05:59:53.235 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-24T05:59:53.235 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-24T05:59:53.235 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-24T05:59:53.235 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-24T05:59:53.235 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-24T05:59:53.236 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-24T05:59:53.236 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-24T05:59:53.236 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-24T05:59:53.236 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-24T05:59:53.236 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-24T05:59:53.236 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-24T05:59:53.237 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-24T05:59:53.237 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-24T05:59:53.237 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-24T05:59:53.238 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-24T05:59:53.238 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-24T05:59:53.238 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-24T05:59:53.238 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-24T05:59:53.238 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-24T05:59:53.239 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-24T05:59:53.239 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-24T05:59:53.239 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-24T05:59:53.239 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-24T05:59:53.239 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-24T05:59:53.239 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-24T05:59:53.240 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-24T05:59:53.240 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-24T05:59:53.240 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-24T05:59:53.240 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-24T05:59:53.240 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-24T05:59:53.241 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-24T05:59:53.241 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-24T05:59:53.241 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-24T05:59:53.241 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-24T05:59:53.241 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-24T05:59:53.241 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-24T05:59:53.242 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-24T05:59:53.242 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-24T05:59:53.242 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-24T05:59:53.242 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-24T05:59:53.243 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-24T05:59:53.243 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-24T05:59:53.243 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-24T05:59:53.243 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-24T05:59:53.244 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-24T05:59:53.244 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-24T05:59:53.244 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-24T05:59:53.244 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-24T05:59:53.244 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-24T05:59:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-24T05:59:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-24T05:59:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-24T05:59:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-24T05:59:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-24T05:59:53.245 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-24T05:59:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-24T05:59:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-24T05:59:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-24T05:59:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-24T05:59:53.246 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-24T05:59:53.247 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-24T05:59:53.247 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-24T05:59:53.247 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-24T05:59:53.247 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-24T05:59:53.247 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-24T05:59:53.247 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-24T05:59:53.248 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-24T05:59:53.249 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-24T05:59:53.249 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-24T05:59:53.249 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-24T05:59:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-24T05:59:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-24T05:59:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-24T05:59:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-24T05:59:53.250 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-24T05:59:53.251 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-24T05:59:53.251 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-24T05:59:53.251 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-24T05:59:53.251 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-24T05:59:53.251 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-24T05:59:53.251 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-24T05:59:53.252 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-24T05:59:53.252 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-24T05:59:53.252 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-24T05:59:53.252 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-24T05:59:53.252 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-24T05:59:53.253 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-24T05:59:53.253 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-24T05:59:53.253 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-24T05:59:53.253 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-24T05:59:53.253 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-24T05:59:53.253 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-24T05:59:53.254 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-24T05:59:53.254 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-24T05:59:53.255 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-24T05:59:53.255 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-24T05:59:53.255 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-24T05:59:53.255 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-24T05:59:53.256 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-24T05:59:53.256 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-24T05:59:53.256 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-24T05:59:53.256 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-24T05:59:53.256 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-24T05:59:53.256 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-24T05:59:53.257 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-24T05:59:53.257 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-24T05:59:53.257 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-24T05:59:53.258 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-24T05:59:53.258 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-24T05:59:53.258 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-24T05:59:53.258 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-24T05:59:53.258 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-24T05:59:53.258 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-24T05:59:53.259 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-24T05:59:53.259 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-24T05:59:53.259 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-24T05:59:53.260 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-24T05:59:53.260 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-24T05:59:53.260 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-24T05:59:53.260 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-24T05:59:53.260 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-24T05:59:53.261 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-24T05:59:53.261 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-24T05:59:53.261 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-24T05:59:53.261 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-24T05:59:53.261 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-24T05:59:53.261 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-24T05:59:53.262 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-24T05:59:53.262 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-24T05:59:53.262 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-24T05:59:53.262 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-24T05:59:53.262 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-24T05:59:53.262 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-24T05:59:53.263 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-24T05:59:53.263 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-24T05:59:53.263 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-24T05:59:53.263 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-24T05:59:53.263 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-24T05:59:53.264 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-24T05:59:53.264 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-24T05:59:53.264 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-24T05:59:53.264 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-24T05:59:53.265 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-24T05:59:53.265 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-24T05:59:53.265 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-24T05:59:53.268 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-24T05:59:53.268 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-24T05:59:53.268 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-24T05:59:53.268 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-24T05:59:53.269 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-24T05:59:53.269 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-24T05:59:53.269 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-24T05:59:53.269 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-24T05:59:53.269 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-24T05:59:53.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-24T05:59:53.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-24T05:59:53.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-24T05:59:53.270 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-24T05:59:53.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-24T05:59:53.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-24T05:59:53.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-24T05:59:53.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-24T05:59:53.271 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-24T05:59:53.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-24T05:59:53.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-24T05:59:53.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-24T05:59:53.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-24T05:59:53.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-24T05:59:53.272 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-24T05:59:53.273 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-24T05:59:53.273 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-24T05:59:53.273 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-24T05:59:53.273 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-24T05:59:53.273 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-24T05:59:53.274 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-24T05:59:53.274 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-24T05:59:53.274 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-24T05:59:53.274 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-24T05:59:53.274 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-24T05:59:53.274 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-24T05:59:53.275 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-24T05:59:53.275 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-24T05:59:53.275 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-24T05:59:53.275 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-24T05:59:53.275 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-24T05:59:53.275 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-24T05:59:53.276 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-24T05:59:53.276 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-24T05:59:53.276 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-24T05:59:53.276 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-24T05:59:53.277 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-24T05:59:53.277 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-24T05:59:53.277 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-24T05:59:53.277 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-24T05:59:53.277 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-24T05:59:53.277 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-24T05:59:53.278 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-24T05:59:53.278 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-24T05:59:53.278 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-24T05:59:53.278 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-24T05:59:53.278 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-24T05:59:53.279 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-24T05:59:53.279 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-24T05:59:53.279 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-24T05:59:53.279 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-24T05:59:53.279 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-24T05:59:53.279 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-24T05:59:53.280 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-24T05:59:53.280 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-24T05:59:53.280 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-24T05:59:53.281 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-24T05:59:53.281 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-24T05:59:53.281 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-24T05:59:53.281 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-24T05:59:53.281 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-24T05:59:53.281 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-24T05:59:53.282 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-24T05:59:53.282 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-24T05:59:53.282 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-24T05:59:53.282 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-24T05:59:53.282 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-24T05:59:53.282 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-24T05:59:53.283 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-24T05:59:53.283 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-24T05:59:53.283 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-24T05:59:53.283 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-24T05:59:53.283 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-24T05:59:53.284 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-24T05:59:53.284 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-24T05:59:53.284 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-24T05:59:53.284 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-24T05:59:53.284 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-24T05:59:53.284 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-24T05:59:53.285 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-24T05:59:53.286 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-24T05:59:53.286 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-24T05:59:53.286 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-24T05:59:53.286 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-24T05:59:53.286 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-24T05:59:53.287 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-24T05:59:53.287 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-24T05:59:53.287 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-24T05:59:53.287 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-24T05:59:53.287 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-24T05:59:53.287 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-24T05:59:53.288 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-24T05:59:53.288 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-24T05:59:53.288 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-24T05:59:53.288 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-24T05:59:53.288 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-24T05:59:53.289 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-24T05:59:53.289 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-24T05:59:53.289 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-24T05:59:53.289 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-24T05:59:53.289 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-24T05:59:53.289 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-24T05:59:53.290 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-24T05:59:53.290 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-24T05:59:53.290 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-24T05:59:53.290 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-24T05:59:53.291 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-24T05:59:53.291 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-24T05:59:53.291 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-24T05:59:53.291 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-24T05:59:53.291 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-24T05:59:53.292 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-24T05:59:53.292 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-24T05:59:53.292 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-24T05:59:53.292 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-24T05:59:53.292 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-24T05:59:53.292 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-24T05:59:53.293 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-24T05:59:53.293 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-24T05:59:53.293 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-24T05:59:53.293 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-24T05:59:53.293 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-24T05:59:53.294 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-24T05:59:53.294 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-24T05:59:53.294 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-24T05:59:53.294 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-24T05:59:53.294 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-24T05:59:53.294 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-24T05:59:53.295 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-24T05:59:53.295 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-24T05:59:53.295 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-24T05:59:53.295 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-24T05:59:53.295 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-24T05:59:53.295 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-24T05:59:53.296 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-24T05:59:53.296 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-24T05:59:53.296 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-24T05:59:53.296 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-24T05:59:53.296 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-24T05:59:53.296 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-24T05:59:53.297 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-24T05:59:53.297 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-24T05:59:53.297 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-24T05:59:53.297 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-24T05:59:53.297 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-24T05:59:53.298 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-24T05:59:53.298 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-24T05:59:53.298 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-24T05:59:53.298 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-24T05:59:53.298 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-24T05:59:53.299 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-24T05:59:53.299 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-24T05:59:53.299 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-24T05:59:53.299 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-24T05:59:53.299 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-24T05:59:53.299 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-24T05:59:53.300 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-24T05:59:53.300 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-24T05:59:53.300 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-24T05:59:53.300 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-24T05:59:53.300 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-24T05:59:53.300 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-24T05:59:53.301 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-24T05:59:53.301 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-24T05:59:53.301 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-24T05:59:53.302 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-24T05:59:53.302 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-24T05:59:53.302 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-24T05:59:53.302 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-24T05:59:53.302 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-24T05:59:53.302 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-24T05:59:53.303 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-24T05:59:53.303 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-24T05:59:53.303 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-24T05:59:53.303 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-24T05:59:53.303 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-24T05:59:53.304 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-24T05:59:53.304 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-24T05:59:53.304 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-24T05:59:53.304 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-24T05:59:53.304 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-24T05:59:53.304 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-24T05:59:53.305 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-24T05:59:53.305 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-24T05:59:53.305 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-24T05:59:53.305 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-24T05:59:53.305 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-24T05:59:53.305 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-24T05:59:53.306 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-24T05:59:53.306 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-24T05:59:53.306 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-24T05:59:53.306 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-24T05:59:53.306 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-24T05:59:53.307 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-24T05:59:53.307 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-24T05:59:53.307 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-24T05:59:53.307 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-24T05:59:53.307 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-24T05:59:53.307 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-24T05:59:53.308 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-24T05:59:53.308 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-24T05:59:53.308 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-24T05:59:53.308 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-24T05:59:53.308 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-24T05:59:53.308 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-24T05:59:53.309 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-24T05:59:53.309 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-24T05:59:53.309 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-24T05:59:53.309 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-24T05:59:53.309 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-24T05:59:53.310 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-24T05:59:53.310 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-24T05:59:53.310 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-24T05:59:53.310 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-24T05:59:53.310 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-24T05:59:53.310 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-24T05:59:53.311 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-24T05:59:53.311 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-24T05:59:53.311 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-24T05:59:53.311 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-24T05:59:53.313 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-24T05:59:53.313 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-24T05:59:53.313 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-24T05:59:53.314 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-24T05:59:53.314 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-24T05:59:53.314 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-24T05:59:53.314 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-24T05:59:53.314 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-24T05:59:53.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-24T05:59:53.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-24T05:59:53.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-24T05:59:53.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-24T05:59:53.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-24T05:59:53.315 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-24T05:59:53.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-24T05:59:53.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-24T05:59:53.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-24T05:59:53.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-24T05:59:53.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-24T05:59:53.316 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-24T05:59:53.317 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-24T05:59:53.317 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-24T05:59:53.317 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-24T05:59:53.317 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-24T05:59:53.317 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-24T05:59:53.318 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-24T05:59:53.318 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-24T05:59:53.318 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-24T05:59:53.318 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-24T05:59:53.318 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-24T05:59:53.318 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-24T05:59:53.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-24T05:59:53.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-24T05:59:53.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-24T05:59:53.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-24T05:59:53.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-24T05:59:53.319 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-24T05:59:53.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-24T05:59:53.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-24T05:59:53.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-24T05:59:53.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-24T05:59:53.320 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-24T05:59:53.321 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-24T05:59:53.321 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-24T05:59:53.321 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-24T05:59:53.321 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-24T05:59:53.321 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-24T05:59:53.321 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-24T05:59:53.322 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-24T05:59:53.322 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-24T05:59:53.322 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-24T05:59:53.322 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-24T05:59:53.322 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-24T05:59:53.323 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-24T05:59:53.323 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-24T05:59:53.323 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-24T05:59:53.323 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-24T05:59:53.324 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-24T05:59:53.324 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-24T05:59:53.324 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-24T05:59:53.324 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-24T05:59:53.324 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-24T05:59:53.325 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-24T05:59:53.325 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-24T05:59:53.325 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-24T05:59:53.325 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-24T05:59:53.325 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-24T05:59:53.325 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-24T05:59:53.326 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-24T05:59:53.326 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-24T05:59:53.326 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-24T05:59:53.326 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-24T05:59:53.326 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-24T05:59:53.326 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-24T05:59:53.327 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-24T05:59:53.327 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-24T05:59:53.327 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-24T05:59:53.327 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-24T05:59:53.328 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-24T05:59:53.328 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-24T05:59:53.328 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-24T05:59:53.328 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-24T05:59:53.328 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-24T05:59:53.328 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-24T05:59:53.329 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-24T05:59:53.329 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-24T05:59:53.329 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-24T05:59:53.329 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-24T05:59:53.329 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-24T05:59:53.330 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-24T05:59:53.330 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-24T05:59:53.330 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-24T05:59:53.330 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-24T05:59:53.330 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-24T05:59:53.330 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-24T05:59:53.331 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-24T05:59:53.331 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-24T05:59:53.331 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-24T05:59:53.331 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-24T05:59:53.331 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-24T05:59:53.331 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-24T05:59:53.332 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-24T05:59:53.332 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-24T05:59:53.332 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-24T05:59:53.332 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-24T05:59:53.332 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-24T05:59:53.333 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-24T05:59:53.333 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-24T05:59:53.333 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-24T05:59:53.333 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-24T05:59:53.333 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-24T05:59:53.333 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-24T05:59:53.334 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-24T05:59:53.334 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-24T05:59:53.334 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-24T05:59:53.335 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-24T05:59:53.335 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-24T05:59:53.335 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-24T05:59:53.335 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-24T05:59:53.335 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-24T05:59:53.335 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-24T05:59:53.336 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-24T05:59:53.336 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-24T05:59:53.336 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-24T05:59:53.336 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-24T05:59:53.336 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-24T05:59:53.336 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-24T05:59:53.337 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-24T05:59:53.337 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-24T05:59:53.337 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-24T05:59:53.337 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-24T05:59:53.337 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-24T05:59:53.338 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-24T05:59:53.338 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-24T05:59:53.338 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-24T05:59:53.338 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-24T05:59:53.338 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-24T05:59:53.338 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-24T05:59:53.339 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-24T05:59:53.339 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-24T05:59:53.339 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-24T05:59:53.339 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-24T05:59:53.339 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-24T05:59:53.339 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-24T05:59:53.340 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-24T05:59:53.340 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-24T05:59:53.340 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-24T05:59:53.340 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-24T05:59:53.340 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-24T05:59:53.340 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-24T05:59:53.341 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-24T05:59:53.341 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-24T05:59:53.341 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-24T05:59:53.341 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-24T05:59:53.341 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-24T05:59:53.342 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-24T05:59:53.342 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-24T05:59:53.342 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-24T05:59:53.342 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-24T05:59:53.342 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-24T05:59:53.342 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-24T05:59:53.343 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-24T05:59:53.343 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-24T05:59:53.343 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-24T05:59:53.343 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-24T05:59:53.343 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-24T05:59:53.344 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-24T05:59:53.344 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-24T05:59:53.344 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-24T05:59:53.345 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-24T05:59:53.345 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-24T05:59:53.345 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-24T05:59:53.345 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-24T05:59:53.345 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-24T05:59:53.346 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-24T05:59:53.346 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-24T05:59:53.346 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-24T05:59:53.346 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-24T05:59:53.346 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-24T05:59:53.346 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-24T05:59:53.347 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-24T05:59:53.347 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-24T05:59:53.347 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-24T05:59:53.347 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-24T05:59:53.347 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-24T05:59:53.348 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-24T05:59:53.348 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-24T05:59:53.348 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-24T05:59:53.348 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-24T05:59:53.349 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-24T05:59:53.349 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-24T05:59:53.349 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-24T05:59:53.349 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-24T05:59:53.349 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-24T05:59:53.349 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-24T05:59:53.350 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-24T05:59:53.350 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-24T05:59:53.350 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-24T05:59:53.350 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-24T05:59:53.350 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-24T05:59:53.351 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-24T05:59:53.351 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-24T05:59:53.351 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-24T05:59:53.351 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-24T05:59:53.351 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-24T05:59:53.351 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-24T05:59:53.352 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-24T05:59:53.352 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-24T05:59:53.352 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-24T05:59:53.352 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-24T05:59:53.352 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-24T05:59:53.352 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-24T05:59:53.353 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-24T05:59:53.353 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-24T05:59:53.353 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-24T05:59:53.353 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-24T05:59:53.353 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-24T05:59:53.354 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-24T05:59:53.354 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-24T05:59:53.354 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-24T05:59:53.354 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-24T05:59:53.354 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-24T05:59:53.354 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-24T05:59:53.355 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-24T05:59:53.355 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-24T05:59:53.356 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-24T05:59:53.356 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-24T05:59:53.356 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-24T05:59:53.356 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-24T05:59:53.356 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-24T05:59:53.356 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-24T05:59:53.357 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-24T05:59:53.357 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-24T05:59:53.357 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-24T05:59:53.357 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-24T05:59:53.357 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-24T05:59:53.357 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-24T05:59:53.358 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-24T05:59:53.358 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-24T05:59:53.358 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-24T05:59:53.358 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-24T05:59:53.358 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-24T05:59:53.359 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-24T05:59:53.359 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-24T05:59:53.359 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-24T05:59:53.359 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-24T05:59:53.359 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-24T05:59:53.359 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-24T05:59:53.360 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-24T05:59:53.360 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-24T05:59:53.360 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-24T05:59:53.360 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-24T05:59:53.360 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-24T05:59:53.360 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-24T05:59:53.361 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-24T05:59:53.361 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-24T05:59:53.361 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-24T05:59:53.361 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-24T05:59:53.361 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-24T05:59:53.362 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-24T05:59:53.362 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-24T05:59:53.362 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-24T05:59:53.362 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-24T05:59:53.362 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-24T05:59:53.362 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-24T05:59:53.363 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-24T05:59:53.363 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-24T05:59:53.363 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-24T05:59:53.363 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-24T05:59:53.363 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-24T05:59:53.364 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-24T05:59:53.364 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-24T05:59:53.364 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-24T05:59:53.364 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-24T05:59:53.364 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-24T05:59:53.364 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-24T05:59:53.365 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-24T05:59:53.365 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-24T05:59:53.365 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-24T05:59:53.365 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-24T05:59:53.365 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-24T05:59:53.366 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-24T05:59:53.366 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-24T05:59:53.366 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-24T05:59:53.366 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-24T05:59:53.367 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-24T05:59:53.367 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-24T05:59:53.367 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-24T05:59:53.367 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-24T05:59:53.367 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-24T05:59:53.367 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-24T05:59:53.368 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-24T05:59:53.368 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-24T05:59:53.368 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-24T05:59:53.368 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-24T05:59:53.369 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-24T05:59:53.369 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-24T05:59:53.369 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-24T05:59:53.369 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-24T05:59:53.369 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-24T05:59:53.369 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-24T05:59:53.370 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-24T05:59:53.370 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-24T05:59:53.370 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-24T05:59:53.370 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-24T05:59:53.370 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-24T05:59:53.371 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-24T05:59:53.371 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-24T05:59:53.371 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-24T05:59:53.371 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-24T05:59:53.371 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-24T05:59:53.371 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-24T05:59:53.372 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-24T05:59:53.372 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-24T05:59:53.372 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-24T05:59:53.372 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-24T05:59:53.372 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-24T05:59:53.372 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-24T05:59:53.373 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-24T05:59:53.373 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-24T05:59:53.373 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-24T05:59:53.373 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-24T05:59:53.373 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-24T05:59:53.374 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-24T05:59:53.374 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-24T05:59:53.374 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-24T05:59:53.374 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-24T05:59:53.374 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-24T05:59:53.374 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-24T05:59:53.375 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-24T05:59:53.375 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-24T05:59:53.375 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-24T05:59:53.375 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-24T05:59:53.375 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-24T05:59:53.375 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-24T05:59:53.376 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-24T05:59:53.376 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-24T05:59:53.376 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-24T05:59:53.377 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-24T05:59:53.377 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-24T05:59:53.377 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-24T05:59:53.377 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-24T05:59:53.377 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-24T05:59:53.378 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-24T05:59:53.378 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-24T05:59:53.378 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-24T05:59:53.378 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-24T05:59:53.378 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-24T05:59:53.379 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-24T05:59:53.379 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-24T05:59:53.379 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-24T05:59:53.379 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-24T05:59:53.379 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-24T05:59:53.379 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-24T05:59:53.380 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-24T05:59:53.380 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-24T05:59:53.380 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-24T05:59:53.380 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-24T05:59:53.380 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-24T05:59:53.381 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-24T05:59:53.381 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-24T05:59:53.381 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-24T05:59:53.381 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-24T05:59:53.381 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-24T05:59:53.381 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-24T05:59:53.382 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-24T05:59:53.382 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-24T05:59:53.382 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-24T05:59:53.382 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-24T05:59:53.382 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-24T05:59:53.383 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-24T05:59:53.383 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-24T05:59:53.383 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-24T05:59:53.383 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-24T05:59:53.383 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-24T05:59:53.383 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-24T05:59:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-24T05:59:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-24T05:59:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-24T05:59:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-24T05:59:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-24T05:59:53.384 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-24T05:59:53.385 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-24T05:59:53.385 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-24T05:59:53.385 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-24T05:59:53.385 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-24T05:59:53.385 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-24T05:59:53.386 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-24T05:59:53.386 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-24T05:59:53.386 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-24T05:59:53.387 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-24T05:59:53.387 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-24T05:59:53.387 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-24T05:59:53.387 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-24T05:59:53.388 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-24T05:59:53.388 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-24T05:59:53.388 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-24T05:59:53.388 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-24T05:59:53.388 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-24T05:59:53.388 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-24T05:59:53.389 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-24T05:59:53.389 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-24T05:59:53.389 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-24T05:59:53.389 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-24T05:59:53.389 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-24T05:59:53.389 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-24T05:59:53.390 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-24T05:59:53.390 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-24T05:59:53.390 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-24T05:59:53.390 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-24T05:59:53.390 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-24T05:59:53.391 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-24T05:59:53.391 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-24T05:59:53.391 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-24T05:59:53.391 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-24T05:59:53.391 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-24T05:59:53.391 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-24T05:59:53.392 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-24T05:59:53.392 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-24T05:59:53.392 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-24T05:59:53.392 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-24T05:59:53.392 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-24T05:59:53.392 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-24T05:59:53.393 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-24T05:59:53.393 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-24T05:59:53.393 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-24T05:59:53.393 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-24T05:59:53.393 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-24T05:59:53.394 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-24T05:59:53.394 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-24T05:59:53.394 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-24T05:59:53.394 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-24T05:59:53.394 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-24T05:59:53.394 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-24T05:59:53.395 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-24T05:59:53.395 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-24T05:59:53.395 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-24T05:59:53.395 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-24T05:59:53.395 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-24T05:59:53.395 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-24T05:59:53.396 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-24T05:59:53.396 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-24T05:59:53.396 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-24T05:59:53.396 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-24T05:59:53.396 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-24T05:59:53.397 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-24T05:59:53.397 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-24T05:59:53.397 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-24T05:59:53.397 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-24T05:59:53.398 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-24T05:59:53.398 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-24T05:59:53.398 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-24T05:59:53.398 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-24T05:59:53.398 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-24T05:59:53.399 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-24T05:59:53.399 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-24T05:59:53.399 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-24T05:59:53.399 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-24T05:59:53.399 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-24T05:59:53.400 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-24T05:59:53.400 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-24T05:59:53.400 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-24T05:59:53.400 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-24T05:59:53.400 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-24T05:59:53.400 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-24T05:59:53.401 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-24T05:59:53.401 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-24T05:59:53.401 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-24T05:59:53.401 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-24T05:59:53.401 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-24T05:59:53.401 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-24T05:59:53.402 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-24T05:59:53.402 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-24T05:59:53.402 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-24T05:59:53.402 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-24T05:59:53.402 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-24T05:59:53.403 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-24T05:59:53.403 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-24T05:59:53.403 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-24T05:59:53.403 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-24T05:59:53.403 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-24T05:59:53.403 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-24T05:59:53.404 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-24T05:59:53.404 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-24T05:59:53.404 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-24T05:59:53.404 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-24T05:59:53.404 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-24T05:59:53.404 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-24T05:59:53.405 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-24T05:59:53.405 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-24T05:59:53.405 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-24T05:59:53.405 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-24T05:59:53.405 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-24T05:59:53.406 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-24T05:59:53.406 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-24T05:59:53.406 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-24T05:59:53.406 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-24T05:59:53.406 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-24T05:59:53.406 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-24T05:59:53.407 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-24T05:59:53.407 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-24T05:59:53.407 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-24T05:59:53.409 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-24T05:59:53.410 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-24T05:59:53.410 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-24T05:59:53.411 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-24T05:59:53.411 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-24T05:59:53.412 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-24T05:59:53.413 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-24T05:59:53.413 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-24T05:59:53.414 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-24T05:59:53.414 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-24T05:59:53.414 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-24T05:59:53.414 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-24T05:59:53.414 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-24T05:59:53.415 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-24T05:59:53.415 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-24T05:59:53.416 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-24T05:59:53.416 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-24T05:59:53.417 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-24T05:59:53.417 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-24T05:59:53.418 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-24T05:59:53.418 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-24T05:59:53.418 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-24T05:59:53.421 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-24T05:59:53.421 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-24T05:59:53.421 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-24T05:59:53.421 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-24T05:59:53.421 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-24T05:59:53.421 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-24T05:59:53.421 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-24T05:59:53.421 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-24T05:59:53.421 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-24T05:59:53.421 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-24T05:59:53.421 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-24T05:59:53.421 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-24T05:59:53.422 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-24T05:59:53.422 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-24T05:59:53.424 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-24T05:59:53.424 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-24T05:59:53.424 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-24T05:59:53.424 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-24T05:59:53.424 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-24T05:59:53.424 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-24T05:59:53.424 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-24T05:59:53.424 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-24T05:59:53.424 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-24T05:59:53.424 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-24T05:59:53.424 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-24T05:59:53.425 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-24T05:59:53.425 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-24T05:59:53.425 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-24T05:59:53.425 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-24T05:59:53.425 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-24T05:59:53.425 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-24T05:59:53.425 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-24T05:59:53.425 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-24T05:59:53.425 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-24T05:59:53.425 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-24T05:59:53.425 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-24T05:59:53.425 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-24T05:59:53.425 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-24T05:59:53.426 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-24T05:59:53.426 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-24T05:59:53.426 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-24T05:59:53.426 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-24T05:59:53.426 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-24T05:59:53.426 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-24T05:59:53.426 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-24T05:59:53.426 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-24T05:59:53.426 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-24T05:59:53.427 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-24T05:59:53.427 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-24T05:59:53.427 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-24T05:59:53.427 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-24T05:59:53.427 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-24T05:59:53.427 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-24T05:59:53.427 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-24T05:59:53.427 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-24T05:59:53.427 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-24T05:59:53.427 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-24T05:59:53.427 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-24T05:59:53.427 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-24T05:59:53.427 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-24T05:59:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-24T05:59:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-24T05:59:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-24T05:59:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-24T05:59:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-24T05:59:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-24T05:59:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-24T05:59:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-24T05:59:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-24T05:59:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-24T05:59:53.428 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-24T05:59:53.429 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-24T05:59:53.429 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-24T05:59:53.429 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-24T05:59:53.429 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-24T05:59:53.429 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-24T05:59:53.429 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-24T05:59:53.429 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-24T05:59:53.429 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-24T05:59:53.429 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-24T05:59:53.430 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-24T05:59:53.430 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-24T05:59:53.430 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-24T05:59:53.430 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-24T05:59:53.430 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-24T05:59:53.430 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-24T05:59:53.430 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-24T05:59:53.430 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-24T05:59:53.430 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-24T05:59:53.430 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-24T05:59:53.430 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-24T05:59:53.430 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-24T05:59:53.431 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-24T05:59:53.431 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-24T05:59:53.431 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-24T05:59:53.431 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-24T05:59:53.431 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-24T05:59:53.431 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-24T05:59:53.431 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-24T05:59:53.431 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-24T05:59:53.431 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-24T05:59:53.431 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-24T05:59:53.431 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-24T05:59:53.431 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-24T05:59:53.432 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-24T05:59:53.432 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-24T05:59:53.432 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-24T05:59:53.432 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-24T05:59:53.432 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-24T05:59:53.432 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-24T05:59:53.432 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-24T05:59:53.432 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-24T05:59:53.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-24T05:59:53.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-24T05:59:53.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-24T05:59:53.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-24T05:59:53.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-24T05:59:53.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-24T05:59:53.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-24T05:59:53.435 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-24T05:59:53.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-24T05:59:53.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-24T05:59:53.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-24T05:59:53.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-24T05:59:53.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-24T05:59:53.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-24T05:59:53.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-24T05:59:53.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-24T05:59:53.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-24T05:59:53.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-24T05:59:53.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-24T05:59:53.436 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-24T05:59:53.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-24T05:59:53.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-24T05:59:53.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-24T05:59:53.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-24T05:59:53.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-24T05:59:53.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-24T05:59:53.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-24T05:59:53.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-24T05:59:53.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-24T05:59:53.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-24T05:59:53.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-24T05:59:53.437 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-24T05:59:53.438 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-24T05:59:53.438 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-24T05:59:53.438 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-24T05:59:53.438 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-24T05:59:53.438 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-24T05:59:53.438 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-24T05:59:53.438 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-24T05:59:53.438 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-24T05:59:53.438 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-24T05:59:53.438 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-24T05:59:53.438 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-24T05:59:53.438 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-24T05:59:53.439 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-24T05:59:53.439 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-24T05:59:53.439 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-24T05:59:53.447 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-24T05:59:53.447 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-24T05:59:53.447 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-24T05:59:53.447 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-24T05:59:53.447 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-24T05:59:53.448 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-24T05:59:53.448 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-24T05:59:53.448 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-24T05:59:53.448 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-24T05:59:53.448 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-24T05:59:53.448 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-24T05:59:53.448 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-24T05:59:53.448 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-24T05:59:53.448 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-24T05:59:53.448 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-24T05:59:53.449 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-24T05:59:53.449 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-24T05:59:53.449 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-24T05:59:53.449 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-24T05:59:53.449 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-24T05:59:53.449 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-24T05:59:53.449 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-24T05:59:53.449 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-24T05:59:53.449 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-24T05:59:53.449 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-24T05:59:53.449 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-24T05:59:53.450 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-24T05:59:53.450 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-24T05:59:53.450 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-24T05:59:53.450 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-24T05:59:53.450 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-24T05:59:53.451 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-24T05:59:53.454 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-24T05:59:53.455 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-24T05:59:53.455 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-24T05:59:53.455 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-24T05:59:53.455 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-24T05:59:53.455 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-24T05:59:53.455 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-24T05:59:53.455 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-24T05:59:53.455 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-24T05:59:53.457 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-24T05:59:53.457 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-24T05:59:53.457 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-24T05:59:53.457 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-24T05:59:53.458 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-24T05:59:53.458 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-24T05:59:53.458 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-24T05:59:53.458 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-24T05:59:53.458 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-24T05:59:53.458 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-24T05:59:53.458 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-24T05:59:53.458 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-24T05:59:53.458 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-24T05:59:53.458 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-24T05:59:53.458 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-24T05:59:53.458 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-24T05:59:53.459 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-24T05:59:53.459 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-24T05:59:53.459 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-24T05:59:53.459 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-24T05:59:53.459 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-24T05:59:53.459 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-24T05:59:53.459 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-24T05:59:53.459 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-24T05:59:53.459 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-24T05:59:53.459 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-24T05:59:53.459 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-24T05:59:53.459 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-24T05:59:53.460 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-24T05:59:53.460 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-24T05:59:53.460 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-24T05:59:53.460 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-24T05:59:53.460 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-24T05:59:53.460 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-24T05:59:53.460 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-24T05:59:53.461 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-24T05:59:53.461 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-24T05:59:53.461 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-24T05:59:53.461 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-24T05:59:53.461 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-24T05:59:53.461 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-24T05:59:53.462 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-24T05:59:53.462 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-24T05:59:53.462 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-24T05:59:53.462 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-24T05:59:53.462 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-24T05:59:53.462 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-24T05:59:53.462 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-24T05:59:53.462 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-24T05:59:53.463 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-24T05:59:53.463 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-24T05:59:53.463 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-24T05:59:53.463 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-24T05:59:53.463 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-24T05:59:53.463 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-24T05:59:53.463 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-24T05:59:53.463 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-24T05:59:53.463 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-24T05:59:53.463 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-24T05:59:53.463 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-24T05:59:53.464 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-24T05:59:53.464 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-24T05:59:53.464 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-24T05:59:53.464 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-24T05:59:53.464 INFO:teuthology.orchestra.run.smithi006.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-24T05:59:53.465 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-24T05:59:53.465 DEBUG:teuthology.orchestra.run.smithi006:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-24T05:59:53.490 INFO:teuthology.orchestra.run.smithi006.stdout:postinstall scriptlet (using /bin/sh): 2023-11-24T05:59:53.491 INFO:teuthology.orchestra.run.smithi006.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-24T05:59:53.491 INFO:teuthology.orchestra.run.smithi006.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:59:53.491 INFO:teuthology.orchestra.run.smithi006.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:59:53.491 INFO:teuthology.orchestra.run.smithi006.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T05:59:53.491 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:59:53.491 INFO:teuthology.orchestra.run.smithi006.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T05:59:53.491 INFO:teuthology.orchestra.run.smithi006.stdout:fi 2023-11-24T05:59:53.491 INFO:teuthology.orchestra.run.smithi006.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-24T05:59:53.491 INFO:teuthology.orchestra.run.smithi006.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-24T05:59:53.491 INFO:teuthology.orchestra.run.smithi006.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-24T05:59:53.491 INFO:teuthology.orchestra.run.smithi006.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-24T05:59:53.491 INFO:teuthology.orchestra.run.smithi006.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-24T05:59:53.492 INFO:teuthology.orchestra.run.smithi006.stdout:fi 2023-11-24T05:59:53.492 INFO:teuthology.orchestra.run.smithi006.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-24T05:59:53.492 INFO:teuthology.orchestra.run.smithi006.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-24T05:59:53.492 INFO:teuthology.orchestra.run.smithi006.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-24T05:59:53.492 INFO:teuthology.orchestra.run.smithi006.stdout:fi 2023-11-24T05:59:53.493 DEBUG:teuthology.orchestra.run.smithi006:> sudo rpm -qi grub2-tools 2023-11-24T05:59:53.579 INFO:teuthology.orchestra.run.smithi006.stdout:Name : grub2-tools 2023-11-24T05:59:53.579 INFO:teuthology.orchestra.run.smithi006.stdout:Epoch : 1 2023-11-24T05:59:53.579 INFO:teuthology.orchestra.run.smithi006.stdout:Version : 2.06 2023-11-24T05:59:53.579 INFO:teuthology.orchestra.run.smithi006.stdout:Release : 68.el9 2023-11-24T05:59:53.579 INFO:teuthology.orchestra.run.smithi006.stdout:Architecture: x86_64 2023-11-24T05:59:53.580 INFO:teuthology.orchestra.run.smithi006.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-24T05:59:53.580 INFO:teuthology.orchestra.run.smithi006.stdout:Group : Unspecified 2023-11-24T05:59:53.581 INFO:teuthology.orchestra.run.smithi006.stdout:Size : 8264254 2023-11-24T05:59:53.581 INFO:teuthology.orchestra.run.smithi006.stdout:License : GPLv3+ 2023-11-24T05:59:53.581 INFO:teuthology.orchestra.run.smithi006.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-24T05:59:53.581 INFO:teuthology.orchestra.run.smithi006.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-24T05:59:53.581 INFO:teuthology.orchestra.run.smithi006.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-24T05:59:53.581 INFO:teuthology.orchestra.run.smithi006.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-24T05:59:53.581 INFO:teuthology.orchestra.run.smithi006.stdout:Packager : builder@centos.org 2023-11-24T05:59:53.581 INFO:teuthology.orchestra.run.smithi006.stdout:Vendor : CentOS 2023-11-24T05:59:53.581 INFO:teuthology.orchestra.run.smithi006.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-24T05:59:53.581 INFO:teuthology.orchestra.run.smithi006.stdout:Summary : Support tools for GRUB. 2023-11-24T05:59:53.582 INFO:teuthology.orchestra.run.smithi006.stdout:Description : 2023-11-24T05:59:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T05:59:53.582 INFO:teuthology.orchestra.run.smithi006.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-24T05:59:53.582 INFO:teuthology.orchestra.run.smithi006.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-24T05:59:53.582 INFO:teuthology.orchestra.run.smithi006.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-24T05:59:53.582 INFO:teuthology.orchestra.run.smithi006.stdout:hardware devices. 2023-11-24T05:59:53.582 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T05:59:53.582 INFO:teuthology.orchestra.run.smithi006.stdout:This subpackage provides tools for support of all platforms. 2023-11-24T05:59:53.582 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-24T05:59:53.583 INFO:teuthology.task.kernel:Updating grub on smithi006 to boot 6.6.0-g8247bbb5b470 2023-11-24T05:59:53.583 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-24T05:59:54.183 INFO:teuthology.orchestra.run.smithi006.stderr:Generating grub configuration file ... 2023-11-24T05:59:55.682 INFO:teuthology.orchestra.run.smithi006.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-24T05:59:55.726 INFO:teuthology.orchestra.run.smithi006.stderr:done 2023-11-24T05:59:55.728 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2023-11-24T05:59:55.744 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.CqenckpHE3 2023-11-24T05:59:55.745 DEBUG:teuthology.orchestra.run.smithi006:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.CqenckpHE3 2023-11-24T05:59:55.831 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.CqenckpHE3 2023-11-24T05:59:55.947 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.CqenckpHE3 is 6KB 2023-11-24T05:59:56.004 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.CqenckpHE3 2023-11-24T05:59:56.021 DEBUG:teuthology.orchestra.run.smithi006:> sudo /bin/ls /boot/loader/entries || true 2023-11-24T05:59:56.088 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-24T05:59:56.088 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-24T05:59:56.088 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-24T05:59:56.088 INFO:teuthology.orchestra.run.smithi006.stdout:15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470.conf 2023-11-24T05:59:56.089 DEBUG:teuthology.orchestra.run.smithi006:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470 2023-11-24T05:59:56.252 DEBUG:teuthology.orchestra.run.smithi006:> sudo shutdown -r now 2023-11-24T06:00:21.966 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-24T06:00:21.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-11-24T06:00:21.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:00:26.258 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-24T06:00:26.258 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-11-24T06:00:26.259 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:00:40.290 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2023-11-24T06:00:44.642 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.6 2023-11-24T06:00:49.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-11-24T06:00:49.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:00:52.358 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2023-11-24T06:00:53.644 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-11-24T06:00:53.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:01:04.369 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-11-24T06:01:04.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:01:36.369 DEBUG:teuthology.orchestra.run.smithi005:> true 2023-11-24T06:01:37.025 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-11-24T06:01:37.025 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.6.0-g8247bbb5b470"... 2023-11-24T06:01:37.026 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2023-11-24T06:01:37.066 INFO:teuthology.orchestra.run.smithi005.stdout:6.6.0-g8247bbb5b470 2023-11-24T06:01:37.066 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-24T06:01:37.066 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-24T06:01:37.066 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-24T06:01:38.067 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-24T06:01:38.068 DEBUG:teuthology.orchestra.run.smithi005:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-24T06:01:38.220 INFO:teuthology.orchestra.run.smithi005.stdout:ttyS1 2023-11-24T06:01:38.263 DEBUG:teuthology.parallel:result is None 2023-11-24T06:01:53.648 DEBUG:teuthology.orchestra.remote:timed out 2023-11-24T06:02:05.649 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-11-24T06:02:05.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:02:06.042 DEBUG:teuthology.orchestra.run.smithi006:> true 2023-11-24T06:02:06.703 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-11-24T06:02:06.703 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.6.0-g8247bbb5b470"... 2023-11-24T06:02:06.703 DEBUG:teuthology.orchestra.run.smithi006:> uname -r 2023-11-24T06:02:06.760 INFO:teuthology.orchestra.run.smithi006.stdout:6.6.0-g8247bbb5b470 2023-11-24T06:02:06.760 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-24T06:02:06.760 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-24T06:02:06.760 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-24T06:02:07.761 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-24T06:02:07.761 DEBUG:teuthology.orchestra.run.smithi006:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-24T06:02:07.929 INFO:teuthology.orchestra.run.smithi006.stdout:ttyS1 2023-11-24T06:02:07.971 DEBUG:teuthology.parallel:result is None 2023-11-24T06:02:07.971 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-24T06:02:07.978 INFO:teuthology.task.internal:Creating test directory... 2023-11-24T06:02:07.979 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T06:02:07.989 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T06:02:08.053 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-24T06:02:08.061 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-24T06:02:08.076 INFO:teuthology.task.internal:Creating archive directory... 2023-11-24T06:02:08.076 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T06:02:08.085 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T06:02:08.165 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-24T06:02:08.172 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-24T06:02:08.172 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T06:02:08.181 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T06:02:08.314 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:02:08.320 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:02:08.363 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:02:08.369 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T06:02:08.374 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-24T06:02:08.383 INFO:teuthology.task.internal:Configuring sudo... 2023-11-24T06:02:08.383 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T06:02:08.389 DEBUG:teuthology.orchestra.run.smithi006:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T06:02:08.491 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-24T06:02:08.500 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-24T06:02:08.501 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T06:02:08.507 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T06:02:08.565 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:02:08.655 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:02:08.783 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:02:08.875 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:02:08.982 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:02:08.982 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T06:02:09.088 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:02:09.182 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T06:02:09.310 DEBUG:teuthology.orchestra.run.smithi006:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:02:09.399 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T06:02:09.506 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:02:09.506 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T06:02:09.606 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2023-11-24T06:02:09.615 DEBUG:teuthology.orchestra.run.smithi006:> sudo service rsyslog restart 2023-11-24T06:02:09.751 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:02:09.796 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:02:10.272 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-24T06:02:10.281 INFO:teuthology.task.internal:Starting timer... 2023-11-24T06:02:10.281 INFO:teuthology.run_tasks:Running task pcp... 2023-11-24T06:02:10.367 INFO:teuthology.run_tasks:Running task selinux... 2023-11-24T06:02:10.376 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2023-11-24T06:02:10.533 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2023-11-24T06:02:10.540 DEBUG:teuthology.orchestra.run.smithi006:> sudo service auditd rotate 2023-11-24T06:02:10.669 INFO:teuthology.orchestra.run.smithi006.stdout:Rotating logs: 2023-11-24T06:02:10.674 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-24T06:02:10.674 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2023-11-24T06:02:10.743 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2023-11-24T06:02:10.744 DEBUG:teuthology.orchestra.run.smithi006:> /usr/sbin/getenforce 2023-11-24T06:02:10.805 INFO:teuthology.orchestra.run.smithi006.stdout:Permissive 2023-11-24T06:02:10.806 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi006.front.sepia.ceph.com': 'permissive'} 2023-11-24T06:02:10.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:02:10.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:02:10.906 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:02:11.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:02:11.009 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-24T06:02:11.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2023-11-24T06:02:11.123 DEBUG:teuthology.orchestra.run.smithi006:> sudo /usr/sbin/setenforce permissive 2023-11-24T06:02:11.236 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-24T06:02:11.245 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-24T06:02:11.245 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-24T06:02:11.265 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-24T06:02:11.266 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi006.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-24T06:12:37.722 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi006.front.sepia.ceph.com')] 2023-11-24T06:12:37.723 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-11-24T06:12:37.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:12:37.860 DEBUG:teuthology.orchestra.run.smithi005:> true 2023-11-24T06:12:38.019 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-11-24T06:12:38.020 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-11-24T06:12:38.020 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi006.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T06:12:38.159 DEBUG:teuthology.orchestra.run.smithi006:> true 2023-11-24T06:12:38.312 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi006.front.sepia.ceph.com' 2023-11-24T06:12:38.312 INFO:teuthology.run_tasks:Running task clock... 2023-11-24T06:12:38.323 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-24T06:12:38.323 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T06:12:38.324 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:12:38.332 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T06:12:38.333 DEBUG:teuthology.orchestra.run.smithi006:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:12:38.441 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T06:12:38.442 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T06:12:38.500 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T06:12:38.501 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T06:12:38.604 INFO:teuthology.orchestra.run.smithi006.stderr:sudo: ntpd: command not found 2023-11-24T06:12:38.607 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2023-11-24T06:12:38.649 INFO:teuthology.orchestra.run.smithi006.stdout:506 Cannot talk to daemon 2023-11-24T06:12:38.651 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2023-11-24T06:12:38.701 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T06:12:38.703 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T06:12:38.757 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T06:12:38.759 INFO:teuthology.orchestra.run.smithi006.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T06:12:38.889 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:12:38.890 INFO:teuthology.orchestra.run.smithi006.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:12:38.897 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:12:38.897 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2023-11-24T06:12:38.898 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:12:38.898 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:12:38.898 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:12:38.898 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:12:38.899 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:12:38.900 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2023-11-24T06:12:38.900 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:12:38.900 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:12:38.900 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:12:38.900 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:12:38.901 INFO:teuthology.run_tasks:Running task install... 2023-11-24T06:12:38.909 DEBUG:teuthology.task.install:project ceph 2023-11-24T06:12:38.909 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}} 2023-11-24T06:12:38.909 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'} 2023-11-24T06:12:38.909 INFO:teuthology.task.install:Using flavor: default 2023-11-24T06:12:38.916 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-24T06:12:38.916 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-24T06:12:38.916 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T06:12:38.917 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:12:38.919 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T06:12:38.919 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:12:39.210 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/ 2023-11-24T06:12:39.210 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T06:12:39.242 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/ 2023-11-24T06:12:39.242 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T06:12:39.319 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T06:12:39.319 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:12:39.319 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T06:12:39.430 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T06:12:39.430 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T06:12:39.468 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T06:12:39.468 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:12:39.468 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T06:12:39.578 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T06:12:39.579 DEBUG:teuthology.orchestra.run.smithi006:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T06:12:39.580 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T06:12:39.724 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T06:12:39.727 DEBUG:teuthology.orchestra.run.smithi006:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T06:12:39.823 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2023-11-24T06:12:39.827 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2023-11-24T06:12:39.872 DEBUG:teuthology.orchestra.run.smithi006:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T06:12:39.969 INFO:teuthology.orchestra.run.smithi006.stdout:check_obsoletes = 1 2023-11-24T06:12:39.974 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum clean all 2023-11-24T06:12:40.196 INFO:teuthology.orchestra.run.smithi005.stdout:73 files removed 2023-11-24T06:12:40.234 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T06:12:40.359 INFO:teuthology.orchestra.run.smithi006.stdout:73 files removed 2023-11-24T06:12:40.402 DEBUG:teuthology.orchestra.run.smithi006:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T06:12:41.174 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 187 kB/s | 78 kB 00:00 2023-11-24T06:12:41.392 INFO:teuthology.orchestra.run.smithi006.stdout:ceph packages for x86_64 168 kB/s | 78 kB 00:00 2023-11-24T06:12:41.674 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 23 kB/s | 9.5 kB 00:00 2023-11-24T06:12:41.926 INFO:teuthology.orchestra.run.smithi006.stdout:ceph noarch packages 22 kB/s | 9.5 kB 00:00 2023-11-24T06:12:42.233 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 3.9 kB/s | 1.9 kB 00:00 2023-11-24T06:12:42.384 INFO:teuthology.orchestra.run.smithi006.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2023-11-24T06:12:42.970 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS-9 - AppStream 9.7 MB/s | 6.4 MB 00:00 2023-11-24T06:12:43.138 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - AppStream 9.4 MB/s | 6.4 MB 00:00 2023-11-24T06:12:44.688 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS-9 - BaseOS 4.3 MB/s | 2.0 MB 00:00 2023-11-24T06:12:44.872 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS-9 - BaseOS 4.2 MB/s | 2.0 MB 00:00 2023-11-24T06:12:45.900 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 7.9 MB 00:00 2023-11-24T06:12:47.868 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 7.9 MB 00:02 2023-11-24T06:12:49.143 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 18 MB 00:01 2023-11-24T06:12:49.963 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - AppStream 22 MB/s | 18 MB 00:00 2023-11-24T06:12:54.164 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - CRB 12 MB/s | 5.8 MB 00:00 2023-11-24T06:12:55.410 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 5.8 MB 00:02 2023-11-24T06:12:56.082 INFO:teuthology.orchestra.run.smithi006.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 15 kB 00:00 2023-11-24T06:12:57.036 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 15 kB 00:00 2023-11-24T06:12:57.084 INFO:teuthology.orchestra.run.smithi006.stdout:Extra Packages for Enterprise Linux 25 MB/s | 20 MB 00:00 2023-11-24T06:12:59.478 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 20 MB 00:02 2023-11-24T06:13:02.950 INFO:teuthology.orchestra.run.smithi006.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2023-11-24T06:13:05.305 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2023-11-24T06:13:05.989 INFO:teuthology.orchestra.run.smithi006.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:13:05.991 INFO:teuthology.orchestra.run.smithi006.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:13:06.011 INFO:teuthology.orchestra.run.smithi006.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-24T06:13:06.013 INFO:teuthology.orchestra.run.smithi006.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-24T06:13:06.014 INFO:teuthology.orchestra.run.smithi006.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-24T06:13:06.016 INFO:teuthology.orchestra.run.smithi006.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-24T06:13:06.018 INFO:teuthology.orchestra.run.smithi006.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-24T06:13:06.022 INFO:teuthology.orchestra.run.smithi006.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-24T06:13:06.024 INFO:teuthology.orchestra.run.smithi006.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-24T06:13:06.114 INFO:teuthology.orchestra.run.smithi006.stdout:Dependencies resolved. 2023-11-24T06:13:06.138 INFO:teuthology.orchestra.run.smithi006.stdout:============================================================================================ 2023-11-24T06:13:06.138 INFO:teuthology.orchestra.run.smithi006.stdout: Package Arch Version Repository Size 2023-11-24T06:13:06.138 INFO:teuthology.orchestra.run.smithi006.stdout:============================================================================================ 2023-11-24T06:13:06.138 INFO:teuthology.orchestra.run.smithi006.stdout:Installing: 2023-11-24T06:13:06.138 INFO:teuthology.orchestra.run.smithi006.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-24T06:13:06.138 INFO:teuthology.orchestra.run.smithi006.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 6.5 k 2023-11-24T06:13:06.138 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 5.2 M 2023-11-24T06:13:06.138 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 859 k 2023-11-24T06:13:06.138 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 141 k 2023-11-24T06:13:06.138 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 1.5 M 2023-11-24T06:13:06.138 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 133 k 2023-11-24T06:13:06.138 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 1.8 M 2023-11-24T06:13:06.139 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 7.4 M 2023-11-24T06:13:06.139 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 50 k 2023-11-24T06:13:06.139 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 8.6 M 2023-11-24T06:13:06.139 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 41 M 2023-11-24T06:13:06.139 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 262 k 2023-11-24T06:13:06.139 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 772 k 2023-11-24T06:13:06.139 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 217 k 2023-11-24T06:13:06.139 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 18 k 2023-11-24T06:13:06.139 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-24T06:13:06.139 INFO:teuthology.orchestra.run.smithi006.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-24T06:13:06.139 INFO:teuthology.orchestra.run.smithi006.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-24T06:13:06.140 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 31 k 2023-11-24T06:13:06.140 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 719 k 2023-11-24T06:13:06.140 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 126 k 2023-11-24T06:13:06.140 INFO:teuthology.orchestra.run.smithi006.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-24T06:13:06.140 INFO:teuthology.orchestra.run.smithi006.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-24T06:13:06.140 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-24T06:13:06.140 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-24T06:13:06.140 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-24T06:13:06.140 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-24T06:13:06.140 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 162 k 2023-11-24T06:13:06.140 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 319 k 2023-11-24T06:13:06.140 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 298 k 2023-11-24T06:13:06.141 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 99 k 2023-11-24T06:13:06.141 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 85 k 2023-11-24T06:13:06.141 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:13:06.141 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 170 k 2023-11-24T06:13:06.141 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-24T06:13:06.141 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrading: 2023-11-24T06:13:06.141 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-24T06:13:06.141 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-24T06:13:06.141 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-24T06:13:06.141 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-24T06:13:06.141 INFO:teuthology.orchestra.run.smithi006.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-24T06:13:06.141 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-24T06:13:06.141 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-24T06:13:06.142 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-24T06:13:06.142 INFO:teuthology.orchestra.run.smithi006.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-24T06:13:06.142 INFO:teuthology.orchestra.run.smithi006.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-24T06:13:06.142 INFO:teuthology.orchestra.run.smithi006.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.3 M 2023-11-24T06:13:06.142 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:13:06.142 INFO:teuthology.orchestra.run.smithi006.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-24T06:13:06.142 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-24T06:13:06.142 INFO:teuthology.orchestra.run.smithi006.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-24T06:13:06.142 INFO:teuthology.orchestra.run.smithi006.stdout:Installing dependencies: 2023-11-24T06:13:06.142 INFO:teuthology.orchestra.run.smithi006.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-24T06:13:06.142 INFO:teuthology.orchestra.run.smithi006.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-24T06:13:06.142 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 19 M 2023-11-24T06:13:06.143 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 28 k 2023-11-24T06:13:06.143 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 2.1 M 2023-11-24T06:13:06.143 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 243 k 2023-11-24T06:13:06.143 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.4 M 2023-11-24T06:13:06.143 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 18 M 2023-11-24T06:13:06.143 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 15 k 2023-11-24T06:13:06.143 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 24 k 2023-11-24T06:13:06.143 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-24T06:13:06.143 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-24T06:13:06.143 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-24T06:13:06.143 INFO:teuthology.orchestra.run.smithi006.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-24T06:13:06.143 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-24T06:13:06.144 INFO:teuthology.orchestra.run.smithi006.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-24T06:13:06.144 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-24T06:13:06.144 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-24T06:13:06.144 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 163 k 2023-11-24T06:13:06.144 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-24T06:13:06.144 INFO:teuthology.orchestra.run.smithi006.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-24T06:13:06.144 INFO:teuthology.orchestra.run.smithi006.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-24T06:13:06.144 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-24T06:13:06.144 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-24T06:13:06.144 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 476 k 2023-11-24T06:13:06.144 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-24T06:13:06.144 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.8 M 2023-11-24T06:13:06.144 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-24T06:13:06.145 INFO:teuthology.orchestra.run.smithi006.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-24T06:13:06.145 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-24T06:13:06.145 INFO:teuthology.orchestra.run.smithi006.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-24T06:13:06.145 INFO:teuthology.orchestra.run.smithi006.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-24T06:13:06.145 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-24T06:13:06.145 INFO:teuthology.orchestra.run.smithi006.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-24T06:13:06.145 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-24T06:13:06.145 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-24T06:13:06.145 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-24T06:13:06.145 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-24T06:13:06.145 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-24T06:13:06.145 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 45 k 2023-11-24T06:13:06.146 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 129 k 2023-11-24T06:13:06.146 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-24T06:13:06.146 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-24T06:13:06.146 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-24T06:13:06.146 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-24T06:13:06.146 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-24T06:13:06.146 INFO:teuthology.orchestra.run.smithi006.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-24T06:13:06.146 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-24T06:13:06.146 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-24T06:13:06.146 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-24T06:13:06.146 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-24T06:13:06.146 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-24T06:13:06.147 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-24T06:13:06.147 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-24T06:13:06.147 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-24T06:13:06.147 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-24T06:13:06.147 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-24T06:13:06.147 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-24T06:13:06.147 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-24T06:13:06.147 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-24T06:13:06.147 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-24T06:13:06.147 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-24T06:13:06.147 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-24T06:13:06.147 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-24T06:13:06.148 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-24T06:13:06.148 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-24T06:13:06.148 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-24T06:13:06.148 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-24T06:13:06.148 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-24T06:13:06.148 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-24T06:13:06.148 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-24T06:13:06.148 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-24T06:13:06.148 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-24T06:13:06.148 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-24T06:13:06.148 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-24T06:13:06.149 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-24T06:13:06.149 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-24T06:13:06.149 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-24T06:13:06.149 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-24T06:13:06.149 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-24T06:13:06.149 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-24T06:13:06.149 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-24T06:13:06.149 INFO:teuthology.orchestra.run.smithi006.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-24T06:13:06.149 INFO:teuthology.orchestra.run.smithi006.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-24T06:13:06.149 INFO:teuthology.orchestra.run.smithi006.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-24T06:13:06.149 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-24T06:13:06.149 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-24T06:13:06.150 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-24T06:13:06.150 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-24T06:13:06.150 INFO:teuthology.orchestra.run.smithi006.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-24T06:13:06.150 INFO:teuthology.orchestra.run.smithi006.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-24T06:13:06.150 INFO:teuthology.orchestra.run.smithi006.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-24T06:13:06.150 INFO:teuthology.orchestra.run.smithi006.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-24T06:13:06.150 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-24T06:13:06.150 INFO:teuthology.orchestra.run.smithi006.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-24T06:13:06.150 INFO:teuthology.orchestra.run.smithi006.stdout:Installing weak dependencies: 2023-11-24T06:13:06.150 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-24T06:13:06.150 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:13:06.150 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction Summary 2023-11-24T06:13:06.151 INFO:teuthology.orchestra.run.smithi006.stdout:============================================================================================ 2023-11-24T06:13:06.151 INFO:teuthology.orchestra.run.smithi006.stdout:Install 132 Packages 2023-11-24T06:13:06.151 INFO:teuthology.orchestra.run.smithi006.stdout:Upgrade 15 Packages 2023-11-24T06:13:06.151 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:13:06.151 INFO:teuthology.orchestra.run.smithi006.stdout:Total download size: 204 M 2023-11-24T06:13:06.151 INFO:teuthology.orchestra.run.smithi006.stdout:Downloading Packages: 2023-11-24T06:13:07.518 INFO:teuthology.orchestra.run.smithi006.stdout:(1/147): ceph-18.0.0-7484.ga13ea5ff.el9.x86_64. 19 kB/s | 6.5 kB 00:00 2023-11-24T06:13:07.927 INFO:teuthology.orchestra.run.smithi006.stdout:(2/147): ceph-fuse-18.0.0-7484.ga13ea5ff.el9.x8 2.1 MB/s | 859 kB 00:00 2023-11-24T06:13:08.019 INFO:teuthology.orchestra.run.smithi006.stdout:(3/147): ceph-immutable-object-cache-18.0.0-748 1.5 MB/s | 141 kB 00:00 2023-11-24T06:13:08.169 INFO:teuthology.orchestra.run.smithi006.stdout:(4/147): ceph-base-18.0.0-7484.ga13ea5ff.el9.x8 5.3 MB/s | 5.2 MB 00:00 2023-11-24T06:13:08.313 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:13:08.315 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T06:13:08.328 INFO:teuthology.orchestra.run.smithi006.stdout:(5/147): ceph-mgr-18.0.0-7484.ga13ea5ff.el9.x86 9.3 MB/s | 1.5 MB 00:00 2023-11-24T06:13:08.336 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-24T06:13:08.337 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-24T06:13:08.339 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-24T06:13:08.340 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-24T06:13:08.343 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-24T06:13:08.347 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-24T06:13:08.348 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-24T06:13:08.435 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2023-11-24T06:13:08.449 INFO:teuthology.orchestra.run.smithi005.stdout:============================================================================================ 2023-11-24T06:13:08.449 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2023-11-24T06:13:08.449 INFO:teuthology.orchestra.run.smithi005.stdout:============================================================================================ 2023-11-24T06:13:08.449 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2023-11-24T06:13:08.450 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-24T06:13:08.450 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 6.5 k 2023-11-24T06:13:08.450 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 5.2 M 2023-11-24T06:13:08.450 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 859 k 2023-11-24T06:13:08.450 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 141 k 2023-11-24T06:13:08.450 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 1.5 M 2023-11-24T06:13:08.450 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 133 k 2023-11-24T06:13:08.450 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 1.8 M 2023-11-24T06:13:08.450 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 7.4 M 2023-11-24T06:13:08.450 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 50 k 2023-11-24T06:13:08.450 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 8.6 M 2023-11-24T06:13:08.450 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 41 M 2023-11-24T06:13:08.450 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 262 k 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 772 k 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 217 k 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 18 k 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 31 k 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 719 k 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 126 k 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-24T06:13:08.451 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-24T06:13:08.452 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 162 k 2023-11-24T06:13:08.452 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 319 k 2023-11-24T06:13:08.452 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 298 k 2023-11-24T06:13:08.452 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 99 k 2023-11-24T06:13:08.452 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 85 k 2023-11-24T06:13:08.452 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:13:08.452 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 170 k 2023-11-24T06:13:08.452 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-24T06:13:08.452 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2023-11-24T06:13:08.452 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-24T06:13:08.452 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-24T06:13:08.452 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-24T06:13:08.453 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-24T06:13:08.453 INFO:teuthology.orchestra.run.smithi005.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-24T06:13:08.453 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-24T06:13:08.453 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-24T06:13:08.453 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-24T06:13:08.453 INFO:teuthology.orchestra.run.smithi005.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-24T06:13:08.453 INFO:teuthology.orchestra.run.smithi005.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-24T06:13:08.453 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.3 M 2023-11-24T06:13:08.454 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T06:13:08.454 INFO:teuthology.orchestra.run.smithi005.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-24T06:13:08.454 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-24T06:13:08.454 INFO:teuthology.orchestra.run.smithi005.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-24T06:13:08.454 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2023-11-24T06:13:08.454 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-24T06:13:08.454 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-24T06:13:08.454 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 19 M 2023-11-24T06:13:08.454 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 28 k 2023-11-24T06:13:08.454 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 2.1 M 2023-11-24T06:13:08.455 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 243 k 2023-11-24T06:13:08.455 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.4 M 2023-11-24T06:13:08.455 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 18 M 2023-11-24T06:13:08.455 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 15 k 2023-11-24T06:13:08.455 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 24 k 2023-11-24T06:13:08.455 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-24T06:13:08.455 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-24T06:13:08.455 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-24T06:13:08.458 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-24T06:13:08.458 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-24T06:13:08.458 INFO:teuthology.orchestra.run.smithi005.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-24T06:13:08.458 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-24T06:13:08.458 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-24T06:13:08.458 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 163 k 2023-11-24T06:13:08.458 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-24T06:13:08.458 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-24T06:13:08.458 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-24T06:13:08.458 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-24T06:13:08.459 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-24T06:13:08.459 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 476 k 2023-11-24T06:13:08.459 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-24T06:13:08.459 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.8 M 2023-11-24T06:13:08.459 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-24T06:13:08.459 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-24T06:13:08.460 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-24T06:13:08.460 INFO:teuthology.orchestra.run.smithi005.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-24T06:13:08.460 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-24T06:13:08.460 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-24T06:13:08.460 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-24T06:13:08.460 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-24T06:13:08.460 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-24T06:13:08.460 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-24T06:13:08.460 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-24T06:13:08.460 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-24T06:13:08.460 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 45 k 2023-11-24T06:13:08.461 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 129 k 2023-11-24T06:13:08.461 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-24T06:13:08.461 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-24T06:13:08.461 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-24T06:13:08.461 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-24T06:13:08.461 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-24T06:13:08.461 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-24T06:13:08.461 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-24T06:13:08.461 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-24T06:13:08.461 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-24T06:13:08.461 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-24T06:13:08.461 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-24T06:13:08.461 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-24T06:13:08.461 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-24T06:13:08.462 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-24T06:13:08.462 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-24T06:13:08.462 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-24T06:13:08.462 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-24T06:13:08.462 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-24T06:13:08.462 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-24T06:13:08.462 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-24T06:13:08.462 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-24T06:13:08.462 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-24T06:13:08.462 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-24T06:13:08.463 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-24T06:13:08.463 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-24T06:13:08.463 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-24T06:13:08.463 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-24T06:13:08.463 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-24T06:13:08.463 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-24T06:13:08.463 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-24T06:13:08.463 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-24T06:13:08.463 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-24T06:13:08.463 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-24T06:13:08.463 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-24T06:13:08.463 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-24T06:13:08.463 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-24T06:13:08.464 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-24T06:13:08.464 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-24T06:13:08.464 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-24T06:13:08.464 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-24T06:13:08.464 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-24T06:13:08.464 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-24T06:13:08.464 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-24T06:13:08.464 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-24T06:13:08.464 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-24T06:13:08.464 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-24T06:13:08.464 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-24T06:13:08.464 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-24T06:13:08.465 INFO:teuthology.orchestra.run.smithi005.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-24T06:13:08.465 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-24T06:13:08.465 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-24T06:13:08.465 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-24T06:13:08.465 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-24T06:13:08.465 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-24T06:13:08.465 INFO:teuthology.orchestra.run.smithi005.stdout:Installing weak dependencies: 2023-11-24T06:13:08.465 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-24T06:13:08.465 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:13:08.465 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2023-11-24T06:13:08.465 INFO:teuthology.orchestra.run.smithi005.stdout:============================================================================================ 2023-11-24T06:13:08.465 INFO:teuthology.orchestra.run.smithi005.stdout:Install 132 Packages 2023-11-24T06:13:08.465 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 15 Packages 2023-11-24T06:13:08.466 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:13:08.466 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 204 M 2023-11-24T06:13:08.466 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2023-11-24T06:13:08.570 INFO:teuthology.orchestra.run.smithi006.stdout:(6/147): ceph-mds-18.0.0-7484.ga13ea5ff.el9.x86 3.8 MB/s | 2.1 MB 00:00 2023-11-24T06:13:08.704 INFO:teuthology.orchestra.run.smithi006.stdout:(7/147): ceph-mon-18.0.0-7484.ga13ea5ff.el9.x86 12 MB/s | 4.4 MB 00:00 2023-11-24T06:13:09.396 INFO:teuthology.orchestra.run.smithi006.stdout:(8/147): ceph-radosgw-18.0.0-7484.ga13ea5ff.el9 13 MB/s | 8.6 MB 00:00 2023-11-24T06:13:09.454 INFO:teuthology.orchestra.run.smithi006.stdout:(9/147): ceph-selinux-18.0.0-7484.ga13ea5ff.el9 421 kB/s | 24 kB 00:00 2023-11-24T06:13:09.524 INFO:teuthology.orchestra.run.smithi005.stdout:(1/147): ceph-18.0.0-7484.ga13ea5ff.el9.x86_64. 20 kB/s | 6.5 kB 00:00 2023-11-24T06:13:10.429 INFO:teuthology.orchestra.run.smithi006.stdout:(10/147): ceph-common-18.0.0-7484.ga13ea5ff.el9 5.8 MB/s | 19 MB 00:03 2023-11-24T06:13:10.533 INFO:teuthology.orchestra.run.smithi006.stdout:(11/147): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 2.1 MB/s | 217 kB 00:00 2023-11-24T06:13:10.622 INFO:teuthology.orchestra.run.smithi006.stdout:(12/147): libcephfs-devel-18.0.0-7484.ga13ea5ff 355 kB/s | 31 kB 00:00 2023-11-24T06:13:10.805 INFO:teuthology.orchestra.run.smithi006.stdout:(13/147): libcephfs2-18.0.0-7484.ga13ea5ff.el9. 3.9 MB/s | 719 kB 00:00 2023-11-24T06:13:10.914 INFO:teuthology.orchestra.run.smithi006.stdout:(14/147): libcephsqlite-18.0.0-7484.ga13ea5ff.e 1.5 MB/s | 163 kB 00:00 2023-11-24T06:13:10.932 INFO:teuthology.orchestra.run.smithi005.stdout:(2/147): ceph-fuse-18.0.0-7484.ga13ea5ff.el9.x8 611 kB/s | 859 kB 00:01 2023-11-24T06:13:11.031 INFO:teuthology.orchestra.run.smithi006.stdout:(15/147): librados-devel-18.0.0-7484.ga13ea5ff. 1.1 MB/s | 126 kB 00:00 2023-11-24T06:13:11.207 INFO:teuthology.orchestra.run.smithi006.stdout:(16/147): libradosstriper1-18.0.0-7484.ga13ea5f 2.7 MB/s | 476 kB 00:00 2023-11-24T06:13:11.216 INFO:teuthology.orchestra.run.smithi005.stdout:(3/147): ceph-immutable-object-cache-18.0.0-748 500 kB/s | 141 kB 00:00 2023-11-24T06:13:11.798 INFO:teuthology.orchestra.run.smithi006.stdout:(17/147): ceph-osd-18.0.0-7484.ga13ea5ff.el9.x8 5.5 MB/s | 18 MB 00:03 2023-11-24T06:13:11.873 INFO:teuthology.orchestra.run.smithi006.stdout:(18/147): python3-ceph-argparse-18.0.0-7484.ga1 613 kB/s | 45 kB 00:00 2023-11-24T06:13:11.982 INFO:teuthology.orchestra.run.smithi006.stdout:(19/147): librgw2-18.0.0-7484.ga13ea5ff.el9.x86 6.2 MB/s | 4.8 MB 00:00 2023-11-24T06:13:12.015 INFO:teuthology.orchestra.run.smithi006.stdout:(20/147): python3-ceph-common-18.0.0-7484.ga13e 914 kB/s | 129 kB 00:00 2023-11-24T06:13:12.066 INFO:teuthology.orchestra.run.smithi006.stdout:(21/147): python3-cephfs-18.0.0-7484.ga13ea5ff. 1.9 MB/s | 162 kB 00:00 2023-11-24T06:13:12.184 INFO:teuthology.orchestra.run.smithi006.stdout:(22/147): python3-rados-18.0.0-7484.ga13ea5ff.e 1.9 MB/s | 319 kB 00:00 2023-11-24T06:13:12.225 INFO:teuthology.orchestra.run.smithi006.stdout:(23/147): python3-rbd-18.0.0-7484.ga13ea5ff.el9 1.8 MB/s | 298 kB 00:00 2023-11-24T06:13:12.258 INFO:teuthology.orchestra.run.smithi006.stdout:(24/147): python3-rgw-18.0.0-7484.ga13ea5ff.el9 1.3 MB/s | 99 kB 00:00 2023-11-24T06:13:12.309 INFO:teuthology.orchestra.run.smithi006.stdout:(25/147): rbd-fuse-18.0.0-7484.ga13ea5ff.el9.x8 1.0 MB/s | 85 kB 00:00 2023-11-24T06:13:12.392 INFO:teuthology.orchestra.run.smithi006.stdout:(26/147): rbd-nbd-18.0.0-7484.ga13ea5ff.el9.x86 2.0 MB/s | 170 kB 00:00 2023-11-24T06:13:12.408 INFO:teuthology.orchestra.run.smithi005.stdout:(4/147): ceph-base-18.0.0-7484.ga13ea5ff.el9.x8 1.6 MB/s | 5.2 MB 00:03 2023-11-24T06:13:12.833 INFO:teuthology.orchestra.run.smithi005.stdout:(5/147): ceph-mgr-18.0.0-7484.ga13ea5ff.el9.x86 3.5 MB/s | 1.5 MB 00:00 2023-11-24T06:13:12.876 INFO:teuthology.orchestra.run.smithi006.stdout:(27/147): ceph-test-18.0.0-7484.ga13ea5ff.el9.x 12 MB/s | 41 MB 00:03 2023-11-24T06:13:12.901 INFO:teuthology.orchestra.run.smithi006.stdout:(28/147): ceph-grafana-dashboards-18.0.0-7484.g 55 kB/s | 28 kB 00:00 2023-11-24T06:13:12.935 INFO:teuthology.orchestra.run.smithi006.stdout:(29/147): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 2.3 MB/s | 133 kB 00:00 2023-11-24T06:13:13.008 INFO:teuthology.orchestra.run.smithi005.stdout:(6/147): ceph-mds-18.0.0-7484.ga13ea5ff.el9.x86 1.2 MB/s | 2.1 MB 00:01 2023-11-24T06:13:13.044 INFO:teuthology.orchestra.run.smithi006.stdout:(30/147): rbd-mirror-18.0.0-7484.ga13ea5ff.el9. 3.8 MB/s | 3.0 MB 00:00 2023-11-24T06:13:13.186 INFO:teuthology.orchestra.run.smithi006.stdout:(31/147): ceph-mgr-dashboard-18.0.0-7484.ga13ea 6.2 MB/s | 1.8 MB 00:00 2023-11-24T06:13:13.227 INFO:teuthology.orchestra.run.smithi006.stdout:(32/147): ceph-mgr-modules-core-18.0.0-7484.ga1 1.3 MB/s | 243 kB 00:00 2023-11-24T06:13:13.270 INFO:teuthology.orchestra.run.smithi006.stdout:(33/147): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 610 kB/s | 50 kB 00:00 2023-11-24T06:13:13.312 INFO:teuthology.orchestra.run.smithi006.stdout:(34/147): ceph-prometheus-alerts-18.0.0-7484.ga 182 kB/s | 15 kB 00:00 2023-11-24T06:13:13.362 INFO:teuthology.orchestra.run.smithi006.stdout:(35/147): ceph-volume-18.0.0-7484.ga13ea5ff.el9 2.8 MB/s | 262 kB 00:00 2023-11-24T06:13:13.462 INFO:teuthology.orchestra.run.smithi006.stdout:(36/147): cephfs-top-18.0.0-7484.ga13ea5ff.el9. 183 kB/s | 18 kB 00:00 2023-11-24T06:13:13.537 INFO:teuthology.orchestra.run.smithi006.stdout:(37/147): cephadm-18.0.0-7484.ga13ea5ff.el9.noa 3.4 MB/s | 772 kB 00:00 2023-11-24T06:13:13.721 INFO:teuthology.orchestra.run.smithi006.stdout:(38/147): ceph-mgr-diskprediction-local-18.0.0- 9.4 MB/s | 7.4 MB 00:00 2023-11-24T06:13:13.783 INFO:teuthology.orchestra.run.smithi005.stdout:(7/147): ceph-common-18.0.0-7484.ga13ea5ff.el9. 4.1 MB/s | 19 MB 00:04 2023-11-24T06:13:13.971 INFO:teuthology.orchestra.run.smithi006.stdout:(39/147): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 920 kB 00:00 2023-11-24T06:13:14.013 INFO:teuthology.orchestra.run.smithi006.stdout:(40/147): automake-1.16.2-8.el9.noarch.rpm 1.2 MB/s | 662 kB 00:00 2023-11-24T06:13:14.030 INFO:teuthology.orchestra.run.smithi006.stdout:(41/147): boost-program-options-1.75.0-8.el9.x8 442 kB/s | 106 kB 00:00 2023-11-24T06:13:14.055 INFO:teuthology.orchestra.run.smithi006.stdout:(42/147): elfutils-libelf-devel-0.190-1.el9.x86 266 kB/s | 21 kB 00:00 2023-11-24T06:13:14.106 INFO:teuthology.orchestra.run.smithi006.stdout:(43/147): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2023-11-24T06:13:14.154 INFO:teuthology.orchestra.run.smithi006.stdout:(44/147): flexiblas-3.0.4-8.el9.x86_64.rpm 331 kB/s | 32 kB 00:00 2023-11-24T06:13:14.175 INFO:teuthology.orchestra.run.smithi005.stdout:(8/147): ceph-mon-18.0.0-7484.ga13ea5ff.el9.x86 3.3 MB/s | 4.4 MB 00:01 2023-11-24T06:13:14.242 INFO:teuthology.orchestra.run.smithi005.stdout:(9/147): ceph-selinux-18.0.0-7484.ga13ea5ff.el9 368 kB/s | 24 kB 00:00 2023-11-24T06:13:14.431 INFO:teuthology.orchestra.run.smithi006.stdout:(45/147): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.0 MB/s | 3.0 MB 00:00 2023-11-24T06:13:14.482 INFO:teuthology.orchestra.run.smithi006.stdout:(46/147): flexiblas-openblas-openmp-3.0.4-8.el9 46 kB/s | 17 kB 00:00 2023-11-24T06:13:14.507 INFO:teuthology.orchestra.run.smithi006.stdout:(47/147): jq-1.6-16.el9.x86_64.rpm 534 kB/s | 186 kB 00:00 2023-11-24T06:13:14.532 INFO:teuthology.orchestra.run.smithi006.stdout:(48/147): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 10 kB 00:00 2023-11-24T06:13:14.557 INFO:teuthology.orchestra.run.smithi006.stdout:(49/147): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 44 kB 00:00 2023-11-24T06:13:14.591 INFO:teuthology.orchestra.run.smithi006.stdout:(50/147): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 159 kB 00:00 2023-11-24T06:13:14.633 INFO:teuthology.orchestra.run.smithi006.stdout:(51/147): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 661 kB 00:00 2023-11-24T06:13:14.666 INFO:teuthology.orchestra.run.smithi006.stdout:(52/147): libtool-2.4.6-45.el9.x86_64.rpm 5.5 MB/s | 578 kB 00:00 2023-11-24T06:13:14.683 INFO:teuthology.orchestra.run.smithi006.stdout:(53/147): libuuid-devel-2.37.4-15.el9.x86_64.rp 292 kB/s | 24 kB 00:00 2023-11-24T06:13:14.709 INFO:teuthology.orchestra.run.smithi006.stdout:(54/147): libzstd-devel-1.5.1-2.el9.x86_64.rpm 639 kB/s | 46 kB 00:00 2023-11-24T06:13:14.742 INFO:teuthology.orchestra.run.smithi005.stdout:(10/147): ceph-radosgw-18.0.0-7484.ga13ea5ff.el 9.0 MB/s | 8.6 MB 00:00 2023-11-24T06:13:14.751 INFO:teuthology.orchestra.run.smithi006.stdout:(55/147): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2023-11-24T06:13:14.792 INFO:teuthology.orchestra.run.smithi005.stdout:(11/147): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 4.4 MB/s | 217 kB 00:00 2023-11-24T06:13:14.810 INFO:teuthology.orchestra.run.smithi006.stdout:(56/147): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.2 MB/s | 282 kB 00:00 2023-11-24T06:13:14.843 INFO:teuthology.orchestra.run.smithi005.stdout:(12/147): libcephfs-devel-18.0.0-7484.ga13ea5ff 636 kB/s | 31 kB 00:00 2023-11-24T06:13:14.843 INFO:teuthology.orchestra.run.smithi006.stdout:(57/147): openblas-0.3.21-2.el9.x86_64.rpm 259 kB/s | 34 kB 00:00 2023-11-24T06:13:14.960 INFO:teuthology.orchestra.run.smithi005.stdout:(13/147): libcephfs2-18.0.0-7484.ga13ea5ff.el9. 6.1 MB/s | 719 kB 00:00 2023-11-24T06:13:15.010 INFO:teuthology.orchestra.run.smithi005.stdout:(14/147): libcephsqlite-18.0.0-7484.ga13ea5ff.e 3.3 MB/s | 163 kB 00:00 2023-11-24T06:13:15.060 INFO:teuthology.orchestra.run.smithi005.stdout:(15/147): librados-devel-18.0.0-7484.ga13ea5ff. 2.5 MB/s | 126 kB 00:00 2023-11-24T06:13:15.144 INFO:teuthology.orchestra.run.smithi005.stdout:(16/147): libradosstriper1-18.0.0-7484.ga13ea5f 5.6 MB/s | 476 kB 00:00 2023-11-24T06:13:15.252 INFO:teuthology.orchestra.run.smithi006.stdout:(58/147): openblas-openmp-0.3.21-2.el9.x86_64.r 9.5 MB/s | 4.7 MB 00:00 2023-11-24T06:13:15.310 INFO:teuthology.orchestra.run.smithi006.stdout:(59/147): postgresql-contrib-13.11-1.el9.x86_64 1.8 MB/s | 811 kB 00:00 2023-11-24T06:13:15.369 INFO:teuthology.orchestra.run.smithi006.stdout:(60/147): postgresql-13.11-1.el9.x86_64.rpm 3.0 MB/s | 1.5 MB 00:00 2023-11-24T06:13:15.394 INFO:teuthology.orchestra.run.smithi006.stdout:(61/147): postgresql-private-libs-13.11-1.el9.x 1.0 MB/s | 134 kB 00:00 2023-11-24T06:13:15.619 INFO:teuthology.orchestra.run.smithi005.stdout:(17/147): librgw2-18.0.0-7484.ga13ea5ff.el9.x86 10 MB/s | 4.8 MB 00:00 2023-11-24T06:13:15.670 INFO:teuthology.orchestra.run.smithi005.stdout:(18/147): python3-ceph-argparse-18.0.0-7484.ga1 926 kB/s | 45 kB 00:00 2023-11-24T06:13:15.695 INFO:teuthology.orchestra.run.smithi006.stdout:(62/147): postgresql-server-13.11-1.el9.x86_64. 15 MB/s | 5.7 MB 00:00 2023-11-24T06:13:15.720 INFO:teuthology.orchestra.run.smithi005.stdout:(19/147): python3-ceph-common-18.0.0-7484.ga13e 2.6 MB/s | 129 kB 00:00 2023-11-24T06:13:15.728 INFO:teuthology.orchestra.run.smithi006.stdout:(63/147): python3-jinja2-2.11.3-4.el9.noarch.rp 722 kB/s | 228 kB 00:00 2023-11-24T06:13:15.778 INFO:teuthology.orchestra.run.smithi005.stdout:(20/147): python3-cephfs-18.0.0-7484.ga13ea5ff. 2.8 MB/s | 162 kB 00:00 2023-11-24T06:13:15.854 INFO:teuthology.orchestra.run.smithi005.stdout:(21/147): python3-rados-18.0.0-7484.ga13ea5ff.e 4.2 MB/s | 319 kB 00:00 2023-11-24T06:13:15.870 INFO:teuthology.orchestra.run.smithi006.stdout:(64/147): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 5.8 MB 00:00 2023-11-24T06:13:15.928 INFO:teuthology.orchestra.run.smithi006.stdout:(65/147): python3-jsonpatch-1.21-16.el9.noarch. 108 kB/s | 24 kB 00:00 2023-11-24T06:13:15.929 INFO:teuthology.orchestra.run.smithi005.stdout:(22/147): python3-rbd-18.0.0-7484.ga13ea5ff.el9 3.9 MB/s | 298 kB 00:00 2023-11-24T06:13:15.962 INFO:teuthology.orchestra.run.smithi006.stdout:(66/147): python3-jsonpointer-2.0-4.el9.noarch. 83 kB/s | 18 kB 00:00 2023-11-24T06:13:15.979 INFO:teuthology.orchestra.run.smithi005.stdout:(23/147): python3-rgw-18.0.0-7484.ga13ea5ff.el9 2.0 MB/s | 99 kB 00:00 2023-11-24T06:13:16.013 INFO:teuthology.orchestra.run.smithi006.stdout:(67/147): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-11-24T06:13:16.029 INFO:teuthology.orchestra.run.smithi005.stdout:(24/147): rbd-fuse-18.0.0-7484.ga13ea5ff.el9.x8 1.7 MB/s | 85 kB 00:00 2023-11-24T06:13:16.038 INFO:teuthology.orchestra.run.smithi006.stdout:(68/147): python3-markupsafe-1.1.1-12.el9.x86_6 429 kB/s | 32 kB 00:00 2023-11-24T06:13:16.172 INFO:teuthology.orchestra.run.smithi006.stdout:(69/147): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 4.8 MB 00:00 2023-11-24T06:13:16.197 INFO:teuthology.orchestra.run.smithi006.stdout:(70/147): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2023-11-24T06:13:16.248 INFO:teuthology.orchestra.run.smithi006.stdout:(71/147): python3-oauthlib-3.1.1-5.el9.noarch.r 872 kB/s | 174 kB 00:00 2023-11-24T06:13:16.297 INFO:teuthology.orchestra.run.smithi006.stdout:(72/147): python3-prettytable-0.7.2-27.el9.noar 867 kB/s | 41 kB 00:00 2023-11-24T06:13:16.331 INFO:teuthology.orchestra.run.smithi006.stdout:(73/147): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2023-11-24T06:13:16.356 INFO:teuthology.orchestra.run.smithi006.stdout:(74/147): python3-pyasn1-modules-0.4.8-6.el9.no 1.9 MB/s | 211 kB 00:00 2023-11-24T06:13:16.380 INFO:teuthology.orchestra.run.smithi005.stdout:(25/147): rbd-mirror-18.0.0-7484.ga13ea5ff.el9. 8.6 MB/s | 3.0 MB 00:00 2023-11-24T06:13:16.381 INFO:teuthology.orchestra.run.smithi006.stdout:(75/147): python3-pytz-2021.1-5.el9.noarch.rpm 594 kB/s | 47 kB 00:00 2023-11-24T06:13:16.423 INFO:teuthology.orchestra.run.smithi006.stdout:(76/147): python3-requests-oauthlib-1.3.0-12.el 656 kB/s | 43 kB 00:00 2023-11-24T06:13:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:(26/147): rbd-nbd-18.0.0-7484.ga13ea5ff.el9.x86 3.4 MB/s | 170 kB 00:00 2023-11-24T06:13:16.472 INFO:teuthology.orchestra.run.smithi005.stdout:(27/147): ceph-grafana-dashboards-18.0.0-7484.g 685 kB/s | 28 kB 00:00 2023-11-24T06:13:16.531 INFO:teuthology.orchestra.run.smithi005.stdout:(28/147): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 2.3 MB/s | 133 kB 00:00 2023-11-24T06:13:16.582 INFO:teuthology.orchestra.run.smithi006.stdout:(77/147): python3-toml-0.10.2-6.el9.noarch.rpm 195 kB/s | 39 kB 00:00 2023-11-24T06:13:16.739 INFO:teuthology.orchestra.run.smithi005.stdout:(29/147): ceph-mgr-dashboard-18.0.0-7484.ga13ea 8.5 MB/s | 1.8 MB 00:00 2023-11-24T06:13:16.799 INFO:teuthology.orchestra.run.smithi006.stdout:(78/147): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2023-11-24T06:13:16.824 INFO:teuthology.orchestra.run.smithi006.stdout:(79/147): socat-1.7.4.1-5.el9.x86_64.rpm 778 kB/s | 300 kB 00:00 2023-11-24T06:13:16.849 INFO:teuthology.orchestra.run.smithi006.stdout:(80/147): uuid-1.6.2-55.el9.x86_64.rpm 215 kB/s | 56 kB 00:00 2023-11-24T06:13:16.916 INFO:teuthology.orchestra.run.smithi006.stdout:(81/147): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.3 MB/s | 85 kB 00:00 2023-11-24T06:13:16.941 INFO:teuthology.orchestra.run.smithi006.stdout:(82/147): zlib-devel-1.2.11-40.el9.x86_64.rpm 490 kB/s | 44 kB 00:00 2023-11-24T06:13:16.966 INFO:teuthology.orchestra.run.smithi006.stdout:(83/147): xmlstarlet-1.6.1-20.el9.x86_64.rpm 543 kB/s | 63 kB 00:00 2023-11-24T06:13:17.000 INFO:teuthology.orchestra.run.smithi006.stdout:(84/147): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2023-11-24T06:13:17.042 INFO:teuthology.orchestra.run.smithi006.stdout:(85/147): libgfortran-11.4.1-2.3.el9.x86_64.rpm 8.0 MB/s | 803 kB 00:00 2023-11-24T06:13:17.075 INFO:teuthology.orchestra.run.smithi006.stdout:(86/147): libquadmath-11.4.1-2.3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2023-11-24T06:13:17.117 INFO:teuthology.orchestra.run.smithi006.stdout:(87/147): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 241 kB 00:00 2023-11-24T06:13:17.159 INFO:teuthology.orchestra.run.smithi006.stdout:(88/147): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.1 MB 00:00 2023-11-24T06:13:17.185 INFO:teuthology.orchestra.run.smithi006.stdout:(89/147): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2023-11-24T06:13:17.263 INFO:teuthology.orchestra.run.smithi006.stdout:(90/147): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2023-11-24T06:13:17.287 INFO:teuthology.orchestra.run.smithi006.stdout:(91/147): python3-requests-2.25.1-7.el9.noarch. 898 kB/s | 113 kB 00:00 2023-11-24T06:13:17.315 INFO:teuthology.orchestra.run.smithi005.stdout:(30/147): ceph-osd-18.0.0-7484.ga13ea5ff.el9.x8 4.1 MB/s | 18 MB 00:04 2023-11-24T06:13:17.321 INFO:teuthology.orchestra.run.smithi006.stdout:(92/147): python3-urllib3-1.26.5-4.el9.noarch.r 1.4 MB/s | 187 kB 00:00 2023-11-24T06:13:17.415 INFO:teuthology.orchestra.run.smithi005.stdout:(31/147): ceph-mgr-modules-core-18.0.0-7484.ga1 2.4 MB/s | 243 kB 00:00 2023-11-24T06:13:17.490 INFO:teuthology.orchestra.run.smithi005.stdout:(32/147): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 679 kB/s | 50 kB 00:00 2023-11-24T06:13:17.632 INFO:teuthology.orchestra.run.smithi005.stdout:(33/147): ceph-mgr-diskprediction-local-18.0.0- 8.3 MB/s | 7.4 MB 00:00 2023-11-24T06:13:17.657 INFO:teuthology.orchestra.run.smithi005.stdout:(34/147): ceph-prometheus-alerts-18.0.0-7484.ga 91 kB/s | 15 kB 00:00 2023-11-24T06:13:17.708 INFO:teuthology.orchestra.run.smithi005.stdout:(35/147): ceph-volume-18.0.0-7484.ga13ea5ff.el9 3.5 MB/s | 262 kB 00:00 2023-11-24T06:13:17.713 INFO:teuthology.orchestra.run.smithi006.stdout:(93/147): gperftools-libs-2.9.1-2.el9.x86_64.rp 791 kB/s | 309 kB 00:00 2023-11-24T06:13:17.766 INFO:teuthology.orchestra.run.smithi005.stdout:(36/147): cephadm-18.0.0-7484.ga13ea5ff.el9.noa 7.0 MB/s | 772 kB 00:00 2023-11-24T06:13:17.791 INFO:teuthology.orchestra.run.smithi005.stdout:(37/147): cephfs-top-18.0.0-7484.ga13ea5ff.el9. 221 kB/s | 18 kB 00:00 2023-11-24T06:13:18.083 INFO:teuthology.orchestra.run.smithi005.stdout:(38/147): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 662 kB 00:00 2023-11-24T06:13:18.142 INFO:teuthology.orchestra.run.smithi005.stdout:(39/147): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 920 kB 00:00 2023-11-24T06:13:18.167 INFO:teuthology.orchestra.run.smithi005.stdout:(40/147): boost-program-options-1.75.0-8.el9.x8 1.3 MB/s | 106 kB 00:00 2023-11-24T06:13:18.192 INFO:teuthology.orchestra.run.smithi005.stdout:(41/147): elfutils-libelf-devel-0.190-1.el9.x86 445 kB/s | 21 kB 00:00 2023-11-24T06:13:18.226 INFO:teuthology.orchestra.run.smithi005.stdout:(42/147): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2023-11-24T06:13:18.251 INFO:teuthology.orchestra.run.smithi005.stdout:(43/147): flexiblas-3.0.4-8.el9.x86_64.rpm 577 kB/s | 32 kB 00:00 2023-11-24T06:13:18.313 INFO:teuthology.orchestra.run.smithi006.stdout:(94/147): python3-devel-3.9.17-2.el9.x86_64.rpm 240 kB/s | 245 kB 00:01 2023-11-24T06:13:18.360 INFO:teuthology.orchestra.run.smithi005.stdout:(44/147): flexiblas-netlib-3.0.4-8.el9.x86_64.r 23 MB/s | 3.0 MB 00:00 2023-11-24T06:13:18.385 INFO:teuthology.orchestra.run.smithi005.stdout:(45/147): flexiblas-openblas-openmp-3.0.4-8.el9 136 kB/s | 17 kB 00:00 2023-11-24T06:13:18.435 INFO:teuthology.orchestra.run.smithi005.stdout:(46/147): jq-1.6-16.el9.x86_64.rpm 3.7 MB/s | 186 kB 00:00 2023-11-24T06:13:18.460 INFO:teuthology.orchestra.run.smithi005.stdout:(47/147): libaio-devel-0.3.111-13.el9.x86_64.rp 136 kB/s | 10 kB 00:00 2023-11-24T06:13:18.511 INFO:teuthology.orchestra.run.smithi005.stdout:(48/147): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 44 kB 00:00 2023-11-24T06:13:18.521 INFO:teuthology.orchestra.run.smithi006.stdout:(95/147): libarrow-doc-9.0.0-9.el9.noarch.rpm 117 kB/s | 24 kB 00:00 2023-11-24T06:13:18.536 INFO:teuthology.orchestra.run.smithi005.stdout:(49/147): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2023-11-24T06:13:18.572 INFO:teuthology.orchestra.run.smithi006.stdout:(96/147): liboath-2.6.7-2.el9.x86_64.rpm 993 kB/s | 49 kB 00:00 2023-11-24T06:13:18.578 INFO:teuthology.orchestra.run.smithi005.stdout:(50/147): librdkafka-1.6.1-102.el9.x86_64.rpm 10 MB/s | 661 kB 00:00 2023-11-24T06:13:18.639 INFO:teuthology.orchestra.run.smithi006.stdout:(97/147): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2023-11-24T06:13:18.652 INFO:teuthology.orchestra.run.smithi005.stdout:(51/147): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 578 kB 00:00 2023-11-24T06:13:18.679 INFO:teuthology.orchestra.run.smithi005.stdout:(52/147): libuuid-devel-2.37.4-15.el9.x86_64.rp 258 kB/s | 24 kB 00:00 2023-11-24T06:13:18.695 INFO:teuthology.orchestra.run.smithi005.stdout:(53/147): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2023-11-24T06:13:18.729 INFO:teuthology.orchestra.run.smithi005.stdout:(54/147): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.6 MB/s | 282 kB 00:00 2023-11-24T06:13:18.762 INFO:teuthology.orchestra.run.smithi005.stdout:(55/147): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2023-11-24T06:13:18.780 INFO:teuthology.orchestra.run.smithi005.stdout:(56/147): openblas-0.3.21-2.el9.x86_64.rpm 722 kB/s | 34 kB 00:00 2023-11-24T06:13:18.930 INFO:teuthology.orchestra.run.smithi005.stdout:(57/147): openblas-openmp-0.3.21-2.el9.x86_64.r 29 MB/s | 4.7 MB 00:00 2023-11-24T06:13:18.947 INFO:teuthology.orchestra.run.smithi006.stdout:(98/147): openssl-devel-3.0.7-25.el9.x86_64.rpm 2.5 MB/s | 4.1 MB 00:01 2023-11-24T06:13:18.980 INFO:teuthology.orchestra.run.smithi005.stdout:(58/147): postgresql-13.11-1.el9.x86_64.rpm 7.6 MB/s | 1.5 MB 00:00 2023-11-24T06:13:18.998 INFO:teuthology.orchestra.run.smithi006.stdout:(99/147): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.3 MB/s | 837 kB 00:00 2023-11-24T06:13:19.022 INFO:teuthology.orchestra.run.smithi005.stdout:(59/147): postgresql-contrib-13.11-1.el9.x86_64 8.7 MB/s | 811 kB 00:00 2023-11-24T06:13:19.048 INFO:teuthology.orchestra.run.smithi005.stdout:(60/147): postgresql-private-libs-13.11-1.el9.x 2.2 MB/s | 134 kB 00:00 2023-11-24T06:13:19.168 INFO:teuthology.orchestra.run.smithi006.stdout:(100/147): libarrow-9.0.0-9.el9.x86_64.rpm 3.0 MB/s | 4.4 MB 00:01 2023-11-24T06:13:19.198 INFO:teuthology.orchestra.run.smithi006.stdout:(101/147): python3-bcrypt-3.2.2-1.el9.x86_64.rp 335 kB/s | 43 kB 00:00 2023-11-24T06:13:19.206 INFO:teuthology.orchestra.run.smithi005.stdout:(61/147): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2023-11-24T06:13:19.223 INFO:teuthology.orchestra.run.smithi006.stdout:(102/147): python3-cachetools-4.2.4-1.el9.noarc 632 kB/s | 32 kB 00:00 2023-11-24T06:13:19.249 INFO:teuthology.orchestra.run.smithi006.stdout:(103/147): python3-certifi-2021.10.8-2.el9.noar 303 kB/s | 15 kB 00:00 2023-11-24T06:13:19.324 INFO:teuthology.orchestra.run.smithi006.stdout:(104/147): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2023-11-24T06:13:19.340 INFO:teuthology.orchestra.run.smithi005.stdout:(62/147): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2023-11-24T06:13:19.357 INFO:teuthology.orchestra.run.smithi005.stdout:(63/147): python3-jinja2-2.11.3-4.el9.noarch.rp 1.7 MB/s | 228 kB 00:00 2023-11-24T06:13:19.399 INFO:teuthology.orchestra.run.smithi005.stdout:(64/147): python3-jsonpatch-1.21-16.el9.noarch. 605 kB/s | 24 kB 00:00 2023-11-24T06:13:19.407 INFO:teuthology.orchestra.run.smithi006.stdout:(105/147): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2023-11-24T06:13:19.424 INFO:teuthology.orchestra.run.smithi005.stdout:(65/147): python3-jsonpointer-2.0-4.el9.noarch. 277 kB/s | 18 kB 00:00 2023-11-24T06:13:19.441 INFO:teuthology.orchestra.run.smithi006.stdout:(106/147): python3-asyncssh-2.13.2-3.el9.noarch 1.1 MB/s | 548 kB 00:00 2023-11-24T06:13:19.474 INFO:teuthology.orchestra.run.smithi005.stdout:(66/147): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-11-24T06:13:19.496 INFO:teuthology.orchestra.run.smithi006.stdout:(107/147): python3-jaraco-8.2.1-3.el9.noarch.rp 122 kB/s | 11 kB 00:00 2023-11-24T06:13:19.499 INFO:teuthology.orchestra.run.smithi005.stdout:(67/147): python3-markupsafe-1.1.1-12.el9.x86_6 428 kB/s | 32 kB 00:00 2023-11-24T06:13:19.525 INFO:teuthology.orchestra.run.smithi006.stdout:(108/147): python3-jaraco-classes-3.2.1-5.el9.n 214 kB/s | 18 kB 00:00 2023-11-24T06:13:19.551 INFO:teuthology.orchestra.run.smithi006.stdout:(109/147): python3-google-auth-2.23.4-1.el9.noa 876 kB/s | 197 kB 00:00 2023-11-24T06:13:19.576 INFO:teuthology.orchestra.run.smithi006.stdout:(110/147): python3-jaraco-collections-3.0.0-8.e 296 kB/s | 23 kB 00:00 2023-11-24T06:13:19.601 INFO:teuthology.orchestra.run.smithi006.stdout:(111/147): python3-jaraco-functools-3.5.0-2.el9 261 kB/s | 19 kB 00:00 2023-11-24T06:13:19.626 INFO:teuthology.orchestra.run.smithi006.stdout:(112/147): python3-jaraco-text-3.2.0-6.el9.noar 264 kB/s | 20 kB 00:00 2023-11-24T06:13:19.633 INFO:teuthology.orchestra.run.smithi005.stdout:(68/147): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2023-11-24T06:13:19.651 INFO:teuthology.orchestra.run.smithi006.stdout:(113/147): python3-jwt+crypto-2.4.0-1.el9.noarc 121 kB/s | 9.0 kB 00:00 2023-11-24T06:13:19.658 INFO:teuthology.orchestra.run.smithi005.stdout:(69/147): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2023-11-24T06:13:19.676 INFO:teuthology.orchestra.run.smithi006.stdout:(114/147): python3-jwt-2.4.0-1.el9.noarch.rpm 550 kB/s | 41 kB 00:00 2023-11-24T06:13:19.702 INFO:teuthology.orchestra.run.smithi006.stdout:(115/147): python3-logutils-0.3.5-21.el9.noarch 947 kB/s | 46 kB 00:00 2023-11-24T06:13:19.708 INFO:teuthology.orchestra.run.smithi005.stdout:(70/147): python3-oauthlib-3.1.1-5.el9.noarch.r 3.5 MB/s | 174 kB 00:00 2023-11-24T06:13:19.733 INFO:teuthology.orchestra.run.smithi005.stdout:(71/147): python3-prettytable-0.7.2-27.el9.noar 559 kB/s | 41 kB 00:00 2023-11-24T06:13:19.735 INFO:teuthology.orchestra.run.smithi006.stdout:(116/147): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2023-11-24T06:13:19.761 INFO:teuthology.orchestra.run.smithi006.stdout:(117/147): python3-natsort-7.1.1-5.el9.noarch.r 1.0 MB/s | 58 kB 00:00 2023-11-24T06:13:19.784 INFO:teuthology.orchestra.run.smithi005.stdout:(72/147): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2023-11-24T06:13:19.809 INFO:teuthology.orchestra.run.smithi005.stdout:(73/147): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2023-11-24T06:13:19.827 INFO:teuthology.orchestra.run.smithi006.stdout:(118/147): python3-portend-3.1.0-2.el9.noarch.r 250 kB/s | 16 kB 00:00 2023-11-24T06:13:19.859 INFO:teuthology.orchestra.run.smithi005.stdout:(74/147): python3-pytz-2021.1-5.el9.noarch.rpm 971 kB/s | 47 kB 00:00 2023-11-24T06:13:19.884 INFO:teuthology.orchestra.run.smithi005.stdout:(75/147): python3-requests-oauthlib-1.3.0-12.el 581 kB/s | 43 kB 00:00 2023-11-24T06:13:19.886 INFO:teuthology.orchestra.run.smithi006.stdout:(119/147): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2023-11-24T06:13:19.935 INFO:teuthology.orchestra.run.smithi005.stdout:(76/147): python3-toml-0.10.2-6.el9.noarch.rpm 794 kB/s | 39 kB 00:00 2023-11-24T06:13:19.936 INFO:teuthology.orchestra.run.smithi006.stdout:(120/147): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2023-11-24T06:13:19.962 INFO:teuthology.orchestra.run.smithi006.stdout:(121/147): python3-repoze-lru-0.7-16.el9.noarch 413 kB/s | 31 kB 00:00 2023-11-24T06:13:20.004 INFO:teuthology.orchestra.run.smithi006.stdout:(122/147): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2023-11-24T06:13:20.046 INFO:teuthology.orchestra.run.smithi006.stdout:(123/147): python3-tempora-5.0.0-2.el9.noarch.r 880 kB/s | 36 kB 00:00 2023-11-24T06:13:20.088 INFO:teuthology.orchestra.run.smithi006.stdout:(124/147): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2023-11-24T06:13:20.137 INFO:teuthology.orchestra.run.smithi006.stdout:(125/147): python3-typing-extensions-4.4.0-2.el 569 kB/s | 51 kB 00:00 2023-11-24T06:13:20.197 INFO:teuthology.orchestra.run.smithi006.stdout:(126/147): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2023-11-24T06:13:20.227 INFO:teuthology.orchestra.run.smithi005.stdout:(77/147): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2023-11-24T06:13:20.252 INFO:teuthology.orchestra.run.smithi005.stdout:(78/147): socat-1.7.4.1-5.el9.x86_64.rpm 984 kB/s | 300 kB 00:00 2023-11-24T06:13:20.255 INFO:teuthology.orchestra.run.smithi006.stdout:(127/147): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2023-11-24T06:13:20.297 INFO:teuthology.orchestra.run.smithi006.stdout:(128/147): python3-zc-lockfile-2.0-10.el9.noarc 492 kB/s | 20 kB 00:00 2023-11-24T06:13:20.302 INFO:teuthology.orchestra.run.smithi005.stdout:(79/147): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2023-11-24T06:13:20.319 INFO:teuthology.orchestra.run.smithi005.stdout:(80/147): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.3 MB/s | 85 kB 00:00 2023-11-24T06:13:20.389 INFO:teuthology.orchestra.run.smithi006.stdout:(129/147): python3-werkzeug-2.0.3-3.el9.1.noarc 2.2 MB/s | 427 kB 00:00 2023-11-24T06:13:20.481 INFO:teuthology.orchestra.run.smithi006.stdout:(130/147): re2-20211101-3.el9.x86_64.rpm 1.0 MB/s | 192 kB 00:00 2023-11-24T06:13:20.485 INFO:teuthology.orchestra.run.smithi005.stdout:(81/147): xmlstarlet-1.6.1-20.el9.x86_64.rpm 379 kB/s | 63 kB 00:00 2023-11-24T06:13:20.519 INFO:teuthology.orchestra.run.smithi005.stdout:(82/147): zlib-devel-1.2.11-40.el9.x86_64.rpm 221 kB/s | 44 kB 00:00 2023-11-24T06:13:20.570 INFO:teuthology.orchestra.run.smithi005.stdout:(83/147): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2023-11-24T06:13:20.603 INFO:teuthology.orchestra.run.smithi005.stdout:(84/147): libgfortran-11.4.1-2.3.el9.x86_64.rpm 9.6 MB/s | 803 kB 00:00 2023-11-24T06:13:20.654 INFO:teuthology.orchestra.run.smithi005.stdout:(85/147): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.9 MB/s | 195 kB 00:00 2023-11-24T06:13:20.670 INFO:teuthology.orchestra.run.smithi005.stdout:(86/147): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2023-11-24T06:13:20.721 INFO:teuthology.orchestra.run.smithi005.stdout:(87/147): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 103 kB 00:00 2023-11-24T06:13:20.771 INFO:teuthology.orchestra.run.smithi005.stdout:(88/147): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2023-11-24T06:13:20.785 INFO:teuthology.orchestra.run.smithi006.stdout:(131/147): python3-kubernetes-26.1.0-2.el9.noar 923 kB/s | 1.0 MB 00:01 2023-11-24T06:13:20.796 INFO:teuthology.orchestra.run.smithi005.stdout:(89/147): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 124 kB 00:00 2023-11-24T06:13:20.821 INFO:teuthology.orchestra.run.smithi005.stdout:(90/147): python3-requests-2.25.1-7.el9.noarch. 2.3 MB/s | 113 kB 00:00 2023-11-24T06:13:20.855 INFO:teuthology.orchestra.run.smithi005.stdout:(91/147): python3-urllib3-1.26.5-4.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2023-11-24T06:13:20.940 INFO:teuthology.orchestra.run.smithi006.stdout:(132/147): thrift-0.15.0-2.el9.x86_64.rpm 2.9 MB/s | 1.6 MB 00:00 2023-11-24T06:13:21.024 INFO:teuthology.orchestra.run.smithi006.stdout:(133/147): librados2-18.0.0-7484.ga13ea5ff.el9. 6.1 MB/s | 3.3 MB 00:00 2023-11-24T06:13:21.049 INFO:teuthology.orchestra.run.smithi006.stdout:(134/147): libacl-devel-2.3.1-4.el9.x86_64.rpm 628 kB/s | 68 kB 00:00 2023-11-24T06:13:21.141 INFO:teuthology.orchestra.run.smithi006.stdout:(135/147): NetworkManager-1.45.6-1.el9.x86_64.r 19 MB/s | 2.2 MB 00:00 2023-11-24T06:13:21.217 INFO:teuthology.orchestra.run.smithi006.stdout:(136/147): NetworkManager-libnm-1.45.6-1.el9.x8 12 MB/s | 1.8 MB 00:00 2023-11-24T06:13:21.233 INFO:teuthology.orchestra.run.smithi006.stdout:(137/147): NetworkManager-team-1.45.6-1.el9.x86 548 kB/s | 42 kB 00:00 2023-11-24T06:13:21.267 INFO:teuthology.orchestra.run.smithi006.stdout:(138/147): NetworkManager-tui-1.45.6-1.el9.x86_ 4.9 MB/s | 245 kB 00:00 2023-11-24T06:13:21.292 INFO:teuthology.orchestra.run.smithi006.stdout:(139/147): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2023-11-24T06:13:21.317 INFO:teuthology.orchestra.run.smithi006.stdout:(140/147): elfutils-debuginfod-client-0.190-1.e 752 kB/s | 36 kB 00:00 2023-11-24T06:13:21.353 INFO:teuthology.orchestra.run.smithi006.stdout:(141/147): elfutils-libelf-0.190-1.el9.x86_64.r 3.3 MB/s | 192 kB 00:00 2023-11-24T06:13:21.385 INFO:teuthology.orchestra.run.smithi006.stdout:(142/147): elfutils-libs-0.190-1.el9.x86_64.rpm 3.9 MB/s | 255 kB 00:00 2023-11-24T06:13:21.443 INFO:teuthology.orchestra.run.smithi006.stdout:(143/147): libacl-2.3.1-4.el9.x86_64.rpm 250 kB/s | 22 kB 00:00 2023-11-24T06:13:21.447 INFO:teuthology.orchestra.run.smithi005.stdout:(92/147): python3-devel-3.9.17-2.el9.x86_64.rpm 415 kB/s | 245 kB 00:00 2023-11-24T06:13:21.477 INFO:teuthology.orchestra.run.smithi006.stdout:(144/147): xfsdump-3.1.12-4.el9.x86_64.rpm 3.5 MB/s | 321 kB 00:00 2023-11-24T06:13:21.569 INFO:teuthology.orchestra.run.smithi006.stdout:(145/147): librbd1-18.0.0-7484.ga13ea5ff.el9.x8 3.9 MB/s | 3.0 MB 00:00 2023-11-24T06:13:21.606 INFO:teuthology.orchestra.run.smithi005.stdout:(93/147): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.9 MB/s | 309 kB 00:00 2023-11-24T06:13:21.931 INFO:teuthology.orchestra.run.smithi005.stdout:(94/147): libarrow-9.0.0-9.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2023-11-24T06:13:21.965 INFO:teuthology.orchestra.run.smithi005.stdout:(95/147): libarrow-doc-9.0.0-9.el9.noarch.rpm 759 kB/s | 24 kB 00:00 2023-11-24T06:13:21.998 INFO:teuthology.orchestra.run.smithi005.stdout:(96/147): liboath-2.6.7-2.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2023-11-24T06:13:22.090 INFO:teuthology.orchestra.run.smithi005.stdout:(97/147): openssl-devel-3.0.7-25.el9.x86_64.rpm 3.3 MB/s | 4.1 MB 00:01 2023-11-24T06:13:22.107 INFO:teuthology.orchestra.run.smithi005.stdout:(98/147): libunwind-1.6.2-1.el9.x86_64.rpm 624 kB/s | 67 kB 00:00 2023-11-24T06:13:22.183 INFO:teuthology.orchestra.run.smithi005.stdout:(99/147): python3-asyncssh-2.13.2-3.el9.noarch. 7.2 MB/s | 548 kB 00:00 2023-11-24T06:13:22.233 INFO:teuthology.orchestra.run.smithi005.stdout:(100/147): python3-bcrypt-3.2.2-1.el9.x86_64.rp 890 kB/s | 43 kB 00:00 2023-11-24T06:13:22.283 INFO:teuthology.orchestra.run.smithi005.stdout:(101/147): python3-cachetools-4.2.4-1.el9.noarc 656 kB/s | 32 kB 00:00 2023-11-24T06:13:22.302 INFO:teuthology.orchestra.run.smithi006.stdout:(146/147): openssl-3.0.7-25.el9.x86_64.rpm 1.4 MB/s | 1.2 MB 00:00 2023-11-24T06:13:22.333 INFO:teuthology.orchestra.run.smithi005.stdout:(102/147): parquet-libs-9.0.0-9.el9.x86_64.rpm 3.4 MB/s | 837 kB 00:00 2023-11-24T06:13:22.358 INFO:teuthology.orchestra.run.smithi005.stdout:(103/147): python3-certifi-2021.10.8-2.el9.noar 201 kB/s | 15 kB 00:00 2023-11-24T06:13:22.392 INFO:teuthology.orchestra.run.smithi005.stdout:(104/147): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2023-11-24T06:13:22.411 INFO:teuthology.orchestra.run.smithi006.stdout:(147/147): openssl-libs-3.0.7-25.el9.x86_64.rpm 2.3 MB/s | 2.2 MB 00:00 2023-11-24T06:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:13:22.426 INFO:teuthology.orchestra.run.smithi006.stdout:Total 13 MB/s | 204 MB 00:16 2023-11-24T06:13:22.442 INFO:teuthology.orchestra.run.smithi005.stdout:(105/147): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2023-11-24T06:13:22.476 INFO:teuthology.orchestra.run.smithi005.stdout:(106/147): python3-google-auth-2.23.4-1.el9.noa 2.3 MB/s | 197 kB 00:00 2023-11-24T06:13:22.501 INFO:teuthology.orchestra.run.smithi005.stdout:(107/147): python3-jaraco-8.2.1-3.el9.noarch.rp 185 kB/s | 11 kB 00:00 2023-11-24T06:13:22.527 INFO:teuthology.orchestra.run.smithi005.stdout:(108/147): python3-jaraco-classes-3.2.1-5.el9.n 356 kB/s | 18 kB 00:00 2023-11-24T06:13:22.552 INFO:teuthology.orchestra.run.smithi005.stdout:(109/147): python3-jaraco-collections-3.0.0-8.e 470 kB/s | 23 kB 00:00 2023-11-24T06:13:22.577 INFO:teuthology.orchestra.run.smithi005.stdout:(110/147): python3-jaraco-functools-3.5.0-2.el9 394 kB/s | 19 kB 00:00 2023-11-24T06:13:22.603 INFO:teuthology.orchestra.run.smithi005.stdout:(111/147): python3-jaraco-text-3.2.0-6.el9.noar 399 kB/s | 20 kB 00:00 2023-11-24T06:13:22.628 INFO:teuthology.orchestra.run.smithi005.stdout:(112/147): python3-jwt+crypto-2.4.0-1.el9.noarc 183 kB/s | 9.0 kB 00:00 2023-11-24T06:13:22.653 INFO:teuthology.orchestra.run.smithi005.stdout:(113/147): python3-jwt-2.4.0-1.el9.noarch.rpm 830 kB/s | 41 kB 00:00 2023-11-24T06:13:22.667 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction check 2023-11-24T06:13:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:(114/147): python3-logutils-0.3.5-21.el9.noarch 944 kB/s | 46 kB 00:00 2023-11-24T06:13:22.754 INFO:teuthology.orchestra.run.smithi005.stdout:(115/147): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2023-11-24T06:13:22.804 INFO:teuthology.orchestra.run.smithi005.stdout:(116/147): python3-more-itertools-8.12.0-2.el9. 793 kB/s | 79 kB 00:00 2023-11-24T06:13:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:(117/147): python3-natsort-7.1.1-5.el9.noarch.r 777 kB/s | 58 kB 00:00 2023-11-24T06:13:22.871 INFO:teuthology.orchestra.run.smithi005.stdout:(118/147): python3-portend-3.1.0-2.el9.noarch.r 403 kB/s | 16 kB 00:00 2023-11-24T06:13:22.905 INFO:teuthology.orchestra.run.smithi005.stdout:(119/147): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2023-11-24T06:13:22.922 INFO:teuthology.orchestra.run.smithi005.stdout:(120/147): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2023-11-24T06:13:22.949 INFO:teuthology.orchestra.run.smithi005.stdout:(121/147): python3-repoze-lru-0.7-16.el9.noarch 709 kB/s | 31 kB 00:00 2023-11-24T06:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction check succeeded. 2023-11-24T06:13:22.965 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction test 2023-11-24T06:13:22.983 INFO:teuthology.orchestra.run.smithi005.stdout:(122/147): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2023-11-24T06:13:23.008 INFO:teuthology.orchestra.run.smithi005.stdout:(123/147): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2023-11-24T06:13:23.033 INFO:teuthology.orchestra.run.smithi005.stdout:(124/147): python3-tempora-5.0.0-2.el9.noarch.r 725 kB/s | 36 kB 00:00 2023-11-24T06:13:23.059 INFO:teuthology.orchestra.run.smithi005.stdout:(125/147): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2023-11-24T06:13:23.092 INFO:teuthology.orchestra.run.smithi005.stdout:(126/147): python3-webob-1.8.7-6.el9.noarch.rpm 3.9 MB/s | 230 kB 00:00 2023-11-24T06:13:23.109 INFO:teuthology.orchestra.run.smithi005.stdout:(127/147): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2023-11-24T06:13:23.593 INFO:teuthology.orchestra.run.smithi005.stdout:(128/147): ceph-test-18.0.0-7484.ga13ea5ff.el9. 4.4 MB/s | 41 MB 00:09 2023-11-24T06:13:23.618 INFO:teuthology.orchestra.run.smithi005.stdout:(129/147): python3-zc-lockfile-2.0-10.el9.noarc 39 kB/s | 20 kB 00:00 2023-11-24T06:13:23.652 INFO:teuthology.orchestra.run.smithi005.stdout:(130/147): python3-werkzeug-2.0.3-3.el9.1.noarc 765 kB/s | 427 kB 00:00 2023-11-24T06:13:23.752 INFO:teuthology.orchestra.run.smithi005.stdout:(131/147): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2023-11-24T06:13:24.019 INFO:teuthology.orchestra.run.smithi005.stdout:(132/147): thrift-0.15.0-2.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2023-11-24T06:13:24.086 INFO:teuthology.orchestra.run.smithi005.stdout:(133/147): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 68 kB 00:00 2023-11-24T06:13:24.187 INFO:teuthology.orchestra.run.smithi005.stdout:(134/147): NetworkManager-1.45.6-1.el9.x86_64.r 23 MB/s | 2.2 MB 00:00 2023-11-24T06:13:24.278 INFO:teuthology.orchestra.run.smithi005.stdout:(135/147): NetworkManager-libnm-1.45.6-1.el9.x8 19 MB/s | 1.8 MB 00:00 2023-11-24T06:13:24.371 INFO:teuthology.orchestra.run.smithi005.stdout:(136/147): librados2-18.0.0-7484.ga13ea5ff.el9. 4.6 MB/s | 3.3 MB 00:00 2023-11-24T06:13:24.396 INFO:teuthology.orchestra.run.smithi005.stdout:(137/147): NetworkManager-team-1.45.6-1.el9.x86 362 kB/s | 42 kB 00:00 2023-11-24T06:13:24.429 INFO:teuthology.orchestra.run.smithi005.stdout:(138/147): NetworkManager-tui-1.45.6-1.el9.x86_ 4.2 MB/s | 245 kB 00:00 2023-11-24T06:13:24.455 INFO:teuthology.orchestra.run.smithi005.stdout:(139/147): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2023-11-24T06:13:24.480 INFO:teuthology.orchestra.run.smithi005.stdout:(140/147): elfutils-debuginfod-client-0.190-1.e 750 kB/s | 36 kB 00:00 2023-11-24T06:13:24.514 INFO:teuthology.orchestra.run.smithi005.stdout:(141/147): elfutils-libelf-0.190-1.el9.x86_64.r 3.3 MB/s | 192 kB 00:00 2023-11-24T06:13:24.555 INFO:teuthology.orchestra.run.smithi005.stdout:(142/147): elfutils-libs-0.190-1.el9.x86_64.rpm 3.4 MB/s | 255 kB 00:00 2023-11-24T06:13:24.581 INFO:teuthology.orchestra.run.smithi005.stdout:(143/147): libacl-2.3.1-4.el9.x86_64.rpm 344 kB/s | 22 kB 00:00 2023-11-24T06:13:24.647 INFO:teuthology.orchestra.run.smithi005.stdout:(144/147): xfsdump-3.1.12-4.el9.x86_64.rpm 3.5 MB/s | 321 kB 00:00 2023-11-24T06:13:24.731 INFO:teuthology.orchestra.run.smithi005.stdout:(145/147): librbd1-18.0.0-7484.ga13ea5ff.el9.x8 3.1 MB/s | 3.0 MB 00:00 2023-11-24T06:13:25.257 INFO:teuthology.orchestra.run.smithi006.stdout:Transaction test succeeded. 2023-11-24T06:13:25.258 INFO:teuthology.orchestra.run.smithi006.stdout:Running transaction 2023-11-24T06:13:25.573 INFO:teuthology.orchestra.run.smithi005.stdout:(146/147): openssl-3.0.7-25.el9.x86_64.rpm 1.2 MB/s | 1.2 MB 00:00 2023-11-24T06:13:25.648 INFO:teuthology.orchestra.run.smithi005.stdout:(147/147): openssl-libs-3.0.7-25.el9.x86_64.rpm 2.2 MB/s | 2.2 MB 00:01 2023-11-24T06:13:25.657 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2023-11-24T06:13:25.657 INFO:teuthology.orchestra.run.smithi005.stdout:Total 12 MB/s | 204 MB 00:17 2023-11-24T06:13:25.890 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2023-11-24T06:13:26.188 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2023-11-24T06:13:26.188 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2023-11-24T06:13:28.379 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2023-11-24T06:13:28.379 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2023-11-24T06:13:32.234 INFO:teuthology.orchestra.run.smithi006.stdout: Preparing : 1/1 2023-11-24T06:13:32.419 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/162 2023-11-24T06:13:32.453 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/162 2023-11-24T06:13:32.526 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/162 2023-11-24T06:13:32.581 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/162 2023-11-24T06:13:32.611 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/162 2023-11-24T06:13:32.721 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/162 2023-11-24T06:13:33.140 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/162 2023-11-24T06:13:33.175 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:13:33.350 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:13:33.385 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:13:33.508 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:13:33.585 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:13:33.656 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/162 2023-11-24T06:13:34.092 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/162 2023-11-24T06:13:34.159 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/162 2023-11-24T06:13:34.182 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:13:34.360 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:13:34.391 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:13:34.484 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:13:34.876 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 16/162 2023-11-24T06:13:34.921 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2023-11-24T06:13:35.005 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:13:35.058 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/162 2023-11-24T06:13:35.094 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:13:35.097 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/162 2023-11-24T06:13:35.178 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/162 2023-11-24T06:13:35.237 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/162 2023-11-24T06:13:35.274 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/162 2023-11-24T06:13:35.333 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:13:35.369 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/162 2023-11-24T06:13:35.391 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:13:35.793 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/162 2023-11-24T06:13:35.821 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:13:35.875 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:13:35.921 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/162 2023-11-24T06:13:36.012 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:13:36.025 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/162 2023-11-24T06:13:36.048 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:13:36.098 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/162 2023-11-24T06:13:36.158 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/162 2023-11-24T06:13:36.173 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:13:36.221 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:13:36.238 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/162 2023-11-24T06:13:36.301 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/162 2023-11-24T06:13:36.310 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/162 2023-11-24T06:13:36.428 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/162 2023-11-24T06:13:36.503 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/162 2023-11-24T06:13:36.545 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/162 2023-11-24T06:13:36.615 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/162 2023-11-24T06:13:36.670 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/162 2023-11-24T06:13:36.678 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/162 2023-11-24T06:13:36.703 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:13:36.724 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 29/162 2023-11-24T06:13:36.760 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 30/162 2023-11-24T06:13:36.865 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T06:13:36.896 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:13:36.908 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 31/162 2023-11-24T06:13:36.996 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T06:13:37.100 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/162 2023-11-24T06:13:37.202 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/162 2023-11-24T06:13:37.349 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 16/162 2023-11-24T06:13:37.518 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:13:37.607 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T06:13:37.813 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:13:37.890 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:13:37.938 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/162 2023-11-24T06:13:38.047 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/162 2023-11-24T06:13:38.128 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/162 2023-11-24T06:13:38.187 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/162 2023-11-24T06:13:38.234 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 38/162 2023-11-24T06:13:38.316 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/162 2023-11-24T06:13:38.376 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T06:13:38.417 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/162 2023-11-24T06:13:38.547 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/162 2023-11-24T06:13:38.562 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/162 2023-11-24T06:13:38.633 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/162 2023-11-24T06:13:38.638 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/162 2023-11-24T06:13:38.673 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/162 2023-11-24T06:13:38.740 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/162 2023-11-24T06:13:38.789 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/162 2023-11-24T06:13:38.803 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/162 2023-11-24T06:13:38.829 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 43/162 2023-11-24T06:13:38.900 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/162 2023-11-24T06:13:38.918 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/162 2023-11-24T06:13:38.974 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/162 2023-11-24T06:13:38.994 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/162 2023-11-24T06:13:39.009 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/162 2023-11-24T06:13:39.029 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/162 2023-11-24T06:13:39.091 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/162 2023-11-24T06:13:39.145 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/162 2023-11-24T06:13:39.181 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 29/162 2023-11-24T06:13:39.185 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/162 2023-11-24T06:13:39.237 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 30/162 2023-11-24T06:13:39.252 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/162 2023-11-24T06:13:39.318 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/162 2023-11-24T06:13:39.349 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 31/162 2023-11-24T06:13:39.523 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/162 2023-11-24T06:13:39.547 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/162 2023-11-24T06:13:39.577 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/162 2023-11-24T06:13:39.617 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/162 2023-11-24T06:13:39.627 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/162 2023-11-24T06:13:39.748 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/162 2023-11-24T06:13:39.800 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/162 2023-11-24T06:13:39.855 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/162 2023-11-24T06:13:39.893 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/162 2023-11-24T06:13:40.061 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 58/162 2023-11-24T06:13:40.116 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/162 2023-11-24T06:13:40.199 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/162 2023-11-24T06:13:40.285 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/162 2023-11-24T06:13:40.338 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/162 2023-11-24T06:13:40.434 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/162 2023-11-24T06:13:40.485 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/162 2023-11-24T06:13:40.539 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/162 2023-11-24T06:13:40.554 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/162 2023-11-24T06:13:40.589 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 38/162 2023-11-24T06:13:40.619 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/162 2023-11-24T06:13:40.659 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/162 2023-11-24T06:13:40.803 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/162 2023-11-24T06:13:40.846 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/162 2023-11-24T06:13:40.870 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/162 2023-11-24T06:13:40.871 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/162 2023-11-24T06:13:40.909 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/162 2023-11-24T06:13:40.943 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/162 2023-11-24T06:13:40.953 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/162 2023-11-24T06:13:41.023 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/162 2023-11-24T06:13:41.068 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/162 2023-11-24T06:13:41.096 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/162 2023-11-24T06:13:41.133 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/162 2023-11-24T06:13:41.148 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 43/162 2023-11-24T06:13:41.202 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/162 2023-11-24T06:13:41.232 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/162 2023-11-24T06:13:41.252 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/162 2023-11-24T06:13:41.331 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/162 2023-11-24T06:13:41.386 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/162 2023-11-24T06:13:41.436 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/162 2023-11-24T06:13:41.480 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/162 2023-11-24T06:13:41.546 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/162 2023-11-24T06:13:41.550 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/162 2023-11-24T06:13:41.592 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/162 2023-11-24T06:13:41.593 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/162 2023-11-24T06:13:41.679 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/162 2023-11-24T06:13:41.754 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/162 2023-11-24T06:13:41.776 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/162 2023-11-24T06:13:41.821 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/162 2023-11-24T06:13:41.840 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/162 2023-11-24T06:13:41.968 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/162 2023-11-24T06:13:42.037 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/162 2023-11-24T06:13:42.045 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/162 2023-11-24T06:13:42.097 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/162 2023-11-24T06:13:42.155 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/162 2023-11-24T06:13:42.297 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/162 2023-11-24T06:13:42.357 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/162 2023-11-24T06:13:42.418 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/162 2023-11-24T06:13:42.455 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/162 2023-11-24T06:13:42.649 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 58/162 2023-11-24T06:13:42.712 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/162 2023-11-24T06:13:42.794 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/162 2023-11-24T06:13:42.881 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/162 2023-11-24T06:13:43.002 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/162 2023-11-24T06:13:43.060 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/162 2023-11-24T06:13:43.093 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/162 2023-11-24T06:13:43.135 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/162 2023-11-24T06:13:43.150 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/162 2023-11-24T06:13:43.214 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/162 2023-11-24T06:13:43.392 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/162 2023-11-24T06:13:43.394 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/162 2023-11-24T06:13:43.439 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/162 2023-11-24T06:13:43.442 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/162 2023-11-24T06:13:43.473 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/162 2023-11-24T06:13:43.513 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/162 2023-11-24T06:13:43.546 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/162 2023-11-24T06:13:43.565 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/162 2023-11-24T06:13:43.610 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/162 2023-11-24T06:13:43.674 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/162 2023-11-24T06:13:43.745 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/162 2023-11-24T06:13:43.812 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/162 2023-11-24T06:13:43.863 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/162 2023-11-24T06:13:43.999 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/162 2023-11-24T06:13:44.040 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/162 2023-11-24T06:13:44.085 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/162 2023-11-24T06:13:44.159 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/162 2023-11-24T06:13:44.212 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/162 2023-11-24T06:13:44.384 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/162 2023-11-24T06:13:44.406 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/162 2023-11-24T06:13:44.425 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/162 2023-11-24T06:13:44.557 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/162 2023-11-24T06:13:44.573 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/162 2023-11-24T06:13:44.641 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/162 2023-11-24T06:13:45.630 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/162 2023-11-24T06:13:45.689 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/162 2023-11-24T06:13:45.723 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/162 2023-11-24T06:13:45.757 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/162 2023-11-24T06:13:46.017 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/162 2023-11-24T06:13:46.052 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/162 2023-11-24T06:13:46.149 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/162 2023-11-24T06:13:46.274 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/162 2023-11-24T06:13:46.369 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/162 2023-11-24T06:13:46.423 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/162 2023-11-24T06:13:46.455 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : jq-1.6-16.el9.x86_64 95/162 2023-11-24T06:13:46.486 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/162 2023-11-24T06:13:46.522 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 97/162 2023-11-24T06:13:46.902 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/162 2023-11-24T06:13:46.951 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:13:46.998 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/162 2023-11-24T06:13:47.116 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:13:47.130 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/162 2023-11-24T06:13:47.149 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 100/162 2023-11-24T06:13:47.207 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/162 2023-11-24T06:13:47.735 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 102/162 2023-11-24T06:13:47.770 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:13:47.871 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:13:47.897 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 104/162 2023-11-24T06:13:48.795 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/162 2023-11-24T06:13:49.361 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/162 2023-11-24T06:13:49.406 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/162 2023-11-24T06:13:49.448 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : jq-1.6-16.el9.x86_64 95/162 2023-11-24T06:13:49.493 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/162 2023-11-24T06:13:49.562 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 97/162 2023-11-24T06:13:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/162 2023-11-24T06:13:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:13:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T06:13:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 100/162 2023-11-24T06:13:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/162 2023-11-24T06:13:50.486 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:13:50.527 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:13:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 102/162 2023-11-24T06:13:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:13:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T06:13:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 104/162 2023-11-24T06:13:51.187 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:13:51.247 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:13:51.419 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:13:51.419 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-24T06:13:51.419 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-24T06:13:51.420 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:13:51.465 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:13:53.724 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:13:53.758 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:13:54.418 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T06:13:54.463 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:13:54.675 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T06:13:54.675 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-24T06:13:54.675 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-24T06:13:54.675 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:13:54.721 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:14:04.860 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:14:04.860 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /sys 2023-11-24T06:14:04.860 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /proc 2023-11-24T06:14:04.860 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /mnt 2023-11-24T06:14:04.860 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /var/tmp 2023-11-24T06:14:04.860 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /home 2023-11-24T06:14:04.860 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /root 2023-11-24T06:14:04.861 INFO:teuthology.orchestra.run.smithi006.stdout:skipping the directory /tmp 2023-11-24T06:14:04.861 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:05.085 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:14:05.138 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:14:05.138 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:05.138 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-24T06:14:05.138 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:06.956 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:14:07.052 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:14:07.052 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:07.053 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-24T06:14:07.053 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:14:07.053 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:14:07.053 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:07.306 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:14:07.384 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:14:07.385 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:07.385 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-24T06:14:07.385 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:14:07.385 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:14:07.385 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:07.911 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T06:14:07.911 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2023-11-24T06:14:07.911 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2023-11-24T06:14:07.911 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2023-11-24T06:14:07.911 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2023-11-24T06:14:07.911 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2023-11-24T06:14:07.912 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /root 2023-11-24T06:14:07.912 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2023-11-24T06:14:07.912 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:07.915 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:14:07.990 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:14:07.990 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:07.990 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-24T06:14:07.990 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:14:07.990 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:14:07.990 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:08.108 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : automake-1.16.2-8.el9.noarch 112/162 2023-11-24T06:14:08.120 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:14:08.174 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T06:14:08.174 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:08.174 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-24T06:14:08.174 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:08.821 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:14:08.885 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:14:08.932 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 114/162 2023-11-24T06:14:09.041 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 115/162 2023-11-24T06:14:09.072 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:14:09.409 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:14:09.439 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:14:09.921 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:14:10.012 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T06:14:10.012 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:10.012 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-24T06:14:10.012 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:14:10.012 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T06:14:10.012 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:10.257 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:14:10.331 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T06:14:10.331 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:10.331 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-24T06:14:10.331 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:14:10.331 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T06:14:10.331 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:10.620 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:14:10.648 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:14:10.796 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:14:10.864 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:14:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T06:14:10.943 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:10.943 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-24T06:14:10.943 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:14:10.944 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T06:14:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:10.973 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 119/162 2023-11-24T06:14:11.017 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:14:11.066 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.2-8.el9.noarch 112/162 2023-11-24T06:14:11.093 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:14:11.094 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:11.094 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-24T06:14:11.094 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:14:11.094 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:14:11.094 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:11.159 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:14:11.203 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:14:11.297 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 122/162 2023-11-24T06:14:11.737 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:14:11.767 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T06:14:11.812 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 114/162 2023-11-24T06:14:11.910 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 115/162 2023-11-24T06:14:11.975 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:14:12.231 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libtool-2.4.6-45.el9.x86_64 123/162 2023-11-24T06:14:12.261 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:14:12.309 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T06:14:12.340 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:14:12.340 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:12.340 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-24T06:14:12.340 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:14:12.340 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:14:12.340 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:12.355 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:14:12.412 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:14:12.487 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:14:12.487 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:12.487 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-24T06:14:12.487 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:12.563 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:14:12.651 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:14:12.651 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:12.651 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-24T06:14:12.651 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:14:12.651 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:14:12.652 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:13.063 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:14:13.142 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:14:13.142 INFO:teuthology.orchestra.run.smithi006.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:13.143 INFO:teuthology.orchestra.run.smithi006.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-24T06:14:13.143 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:14:13.143 INFO:teuthology.orchestra.run.smithi006.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:14:13.143 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T06:14:13.589 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:14:13.744 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T06:14:13.930 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 119/162 2023-11-24T06:14:13.958 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:14:14.038 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T06:14:14.039 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:14.039 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-24T06:14:14.039 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:14:14.039 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T06:14:14.039 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:14.092 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:14:14.133 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T06:14:14.196 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 122/162 2023-11-24T06:14:15.136 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-45.el9.x86_64 123/162 2023-11-24T06:14:15.169 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:14:15.246 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T06:14:15.246 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:15.246 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-24T06:14:15.246 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:14:15.246 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T06:14:15.246 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:15.295 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:14:15.370 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T06:14:15.370 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:15.370 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-24T06:14:15.370 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:15.437 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:14:15.519 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T06:14:15.519 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:15.519 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-24T06:14:15.519 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:14:15.519 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T06:14:15.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:15.921 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:14:16.002 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T06:14:16.003 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T06:14:16.003 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-24T06:14:16.003 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:14:16.003 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T06:14:16.003 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:17.555 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 128/162 2023-11-24T06:14:17.614 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 129/162 2023-11-24T06:14:17.671 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 130/162 2023-11-24T06:14:17.710 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/162 2023-11-24T06:14:18.008 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 132/162 2023-11-24T06:14:18.131 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 133/162 2023-11-24T06:14:18.198 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 134/162 2023-11-24T06:14:18.231 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 135/162 2023-11-24T06:14:18.271 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 136/162 2023-11-24T06:14:18.318 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 137/162 2023-11-24T06:14:18.423 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 138/162 2023-11-24T06:14:18.491 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 139/162 2023-11-24T06:14:18.566 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 140/162 2023-11-24T06:14:19.488 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:14:19.541 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:14:19.600 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:14:20.412 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 128/162 2023-11-24T06:14:20.470 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 129/162 2023-11-24T06:14:20.534 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 130/162 2023-11-24T06:14:20.569 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/162 2023-11-24T06:14:20.893 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 132/162 2023-11-24T06:14:21.023 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 133/162 2023-11-24T06:14:21.050 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 142/162 2023-11-24T06:14:21.056 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 134/162 2023-11-24T06:14:21.089 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 135/162 2023-11-24T06:14:21.129 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 136/162 2023-11-24T06:14:21.165 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 137/162 2023-11-24T06:14:21.197 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 143/162 2023-11-24T06:14:21.253 INFO:teuthology.orchestra.run.smithi006.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 144/162 2023-11-24T06:14:21.301 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 138/162 2023-11-24T06:14:21.331 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 145/162 2023-11-24T06:14:21.371 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 139/162 2023-11-24T06:14:21.417 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 140/162 2023-11-24T06:14:21.509 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : flex-2.6.4-9.el9.x86_64 146/162 2023-11-24T06:14:21.549 INFO:teuthology.orchestra.run.smithi006.stdout: Installing : bison-3.7.4-5.el9.x86_64 147/162 2023-11-24T06:14:21.600 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/162 2023-11-24T06:14:21.634 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/162 2023-11-24T06:14:21.676 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 150/162 2023-11-24T06:14:21.677 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:14:21.833 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:14:21.884 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 152/162 2023-11-24T06:14:21.929 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 153/162 2023-11-24T06:14:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 154/162 2023-11-24T06:14:21.962 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:14:22.133 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:14:22.164 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-11-24T06:14:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:14:22.205 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:14:22.402 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:14:22.473 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:14:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:14:22.523 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:14:22.545 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T06:14:22.701 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:14:22.749 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 159/162 2023-11-24T06:14:22.783 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 160/162 2023-11-24T06:14:22.828 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 161/162 2023-11-24T06:14:22.958 INFO:teuthology.orchestra.run.smithi006.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:14:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:14:23.235 INFO:teuthology.orchestra.run.smithi006.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-24T06:14:23.235 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:24.011 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 142/162 2023-11-24T06:14:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 143/162 2023-11-24T06:14:24.205 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 144/162 2023-11-24T06:14:24.304 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 145/162 2023-11-24T06:14:24.491 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.4-9.el9.x86_64 146/162 2023-11-24T06:14:24.526 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.7.4-5.el9.x86_64 147/162 2023-11-24T06:14:24.559 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/162 2023-11-24T06:14:24.594 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/162 2023-11-24T06:14:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 150/162 2023-11-24T06:14:24.635 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:14:24.751 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T06:14:24.833 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 152/162 2023-11-24T06:14:24.890 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 153/162 2023-11-24T06:14:24.922 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 154/162 2023-11-24T06:14:24.922 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:14:25.060 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T06:14:25.091 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-11-24T06:14:25.131 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:14:25.132 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:14:25.482 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T06:14:25.482 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:14:25.624 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T06:14:25.660 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 159/162 2023-11-24T06:14:25.702 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 160/162 2023-11-24T06:14:25.745 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 161/162 2023-11-24T06:14:25.878 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:14:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T06:14:26.171 INFO:teuthology.orchestra.run.smithi005.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-24T06:14:26.171 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:27.943 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 1/162 2023-11-24T06:14:27.943 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2/162 2023-11-24T06:14:27.943 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 3/162 2023-11-24T06:14:27.943 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 4/162 2023-11-24T06:14:27.943 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/162 2023-11-24T06:14:27.943 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 6/162 2023-11-24T06:14:27.944 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 7/162 2023-11-24T06:14:27.944 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:14:27.944 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:14:27.944 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:14:27.944 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 11/162 2023-11-24T06:14:27.944 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 12/162 2023-11-24T06:14:27.944 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 13/162 2023-11-24T06:14:27.944 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 14/162 2023-11-24T06:14:27.944 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 15/162 2023-11-24T06:14:27.944 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 16/162 2023-11-24T06:14:27.944 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 17/162 2023-11-24T06:14:27.945 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 18/162 2023-11-24T06:14:27.945 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 19/162 2023-11-24T06:14:27.945 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/162 2023-11-24T06:14:27.945 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 21/162 2023-11-24T06:14:27.945 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 22/162 2023-11-24T06:14:27.945 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 23/162 2023-11-24T06:14:27.945 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 24/162 2023-11-24T06:14:27.945 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 25/162 2023-11-24T06:14:27.957 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 26/162 2023-11-24T06:14:27.957 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 27/162 2023-11-24T06:14:27.957 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 28/162 2023-11-24T06:14:27.957 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/162 2023-11-24T06:14:27.957 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 30/162 2023-11-24T06:14:27.957 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 31/162 2023-11-24T06:14:27.957 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/162 2023-11-24T06:14:27.957 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/162 2023-11-24T06:14:27.957 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 34/162 2023-11-24T06:14:27.957 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/162 2023-11-24T06:14:27.957 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 36/162 2023-11-24T06:14:27.958 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 37/162 2023-11-24T06:14:27.958 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 38/162 2023-11-24T06:14:27.958 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/162 2023-11-24T06:14:27.958 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/162 2023-11-24T06:14:27.958 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/162 2023-11-24T06:14:27.958 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/162 2023-11-24T06:14:27.958 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/162 2023-11-24T06:14:27.958 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/162 2023-11-24T06:14:27.958 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/162 2023-11-24T06:14:27.958 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/162 2023-11-24T06:14:27.958 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : jq-1.6-16.el9.x86_64 47/162 2023-11-24T06:14:27.958 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/162 2023-11-24T06:14:27.959 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/162 2023-11-24T06:14:27.959 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/162 2023-11-24T06:14:27.959 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/162 2023-11-24T06:14:27.959 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/162 2023-11-24T06:14:27.959 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/162 2023-11-24T06:14:27.959 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/162 2023-11-24T06:14:27.959 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/162 2023-11-24T06:14:27.959 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/162 2023-11-24T06:14:27.959 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/162 2023-11-24T06:14:27.959 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/162 2023-11-24T06:14:27.959 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/162 2023-11-24T06:14:27.959 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/162 2023-11-24T06:14:27.959 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/162 2023-11-24T06:14:27.960 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/162 2023-11-24T06:14:27.960 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/162 2023-11-24T06:14:27.960 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/162 2023-11-24T06:14:27.960 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/162 2023-11-24T06:14:27.960 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/162 2023-11-24T06:14:27.960 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/162 2023-11-24T06:14:27.960 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/162 2023-11-24T06:14:27.960 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/162 2023-11-24T06:14:27.960 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/162 2023-11-24T06:14:27.960 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/162 2023-11-24T06:14:27.960 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/162 2023-11-24T06:14:27.960 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/162 2023-11-24T06:14:27.960 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/162 2023-11-24T06:14:27.960 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/162 2023-11-24T06:14:27.961 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/162 2023-11-24T06:14:27.961 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/162 2023-11-24T06:14:27.961 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/162 2023-11-24T06:14:27.961 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/162 2023-11-24T06:14:27.961 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/162 2023-11-24T06:14:27.961 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/162 2023-11-24T06:14:27.961 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/162 2023-11-24T06:14:27.961 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/162 2023-11-24T06:14:27.961 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/162 2023-11-24T06:14:27.961 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/162 2023-11-24T06:14:27.961 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/162 2023-11-24T06:14:27.961 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/162 2023-11-24T06:14:27.961 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/162 2023-11-24T06:14:27.962 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/162 2023-11-24T06:14:27.962 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/162 2023-11-24T06:14:27.962 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/162 2023-11-24T06:14:27.962 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/162 2023-11-24T06:14:27.962 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/162 2023-11-24T06:14:27.962 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/162 2023-11-24T06:14:27.962 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/162 2023-11-24T06:14:27.962 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/162 2023-11-24T06:14:27.962 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/162 2023-11-24T06:14:27.962 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/162 2023-11-24T06:14:27.962 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/162 2023-11-24T06:14:27.962 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/162 2023-11-24T06:14:27.962 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/162 2023-11-24T06:14:27.963 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/162 2023-11-24T06:14:27.963 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/162 2023-11-24T06:14:27.963 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/162 2023-11-24T06:14:27.963 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/162 2023-11-24T06:14:27.963 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/162 2023-11-24T06:14:27.963 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/162 2023-11-24T06:14:27.963 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/162 2023-11-24T06:14:27.963 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/162 2023-11-24T06:14:27.963 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/162 2023-11-24T06:14:27.963 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/162 2023-11-24T06:14:27.963 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/162 2023-11-24T06:14:27.963 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/162 2023-11-24T06:14:27.963 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/162 2023-11-24T06:14:27.963 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/162 2023-11-24T06:14:27.964 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/162 2023-11-24T06:14:27.964 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/162 2023-11-24T06:14:27.964 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/162 2023-11-24T06:14:27.964 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/162 2023-11-24T06:14:27.964 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/162 2023-11-24T06:14:27.964 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/162 2023-11-24T06:14:27.964 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/162 2023-11-24T06:14:27.964 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/162 2023-11-24T06:14:27.964 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/162 2023-11-24T06:14:27.964 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/162 2023-11-24T06:14:27.964 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/162 2023-11-24T06:14:27.964 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/162 2023-11-24T06:14:27.964 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/162 2023-11-24T06:14:27.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/162 2023-11-24T06:14:27.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/162 2023-11-24T06:14:27.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/162 2023-11-24T06:14:27.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/162 2023-11-24T06:14:27.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 133/162 2023-11-24T06:14:27.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/162 2023-11-24T06:14:27.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 135/162 2023-11-24T06:14:27.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/162 2023-11-24T06:14:27.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-11-24T06:14:27.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/162 2023-11-24T06:14:27.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/162 2023-11-24T06:14:27.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/162 2023-11-24T06:14:27.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/162 2023-11-24T06:14:27.965 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/162 2023-11-24T06:14:27.966 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/162 2023-11-24T06:14:27.966 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/162 2023-11-24T06:14:27.966 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/162 2023-11-24T06:14:27.966 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/162 2023-11-24T06:14:27.966 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/162 2023-11-24T06:14:27.966 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/162 2023-11-24T06:14:27.966 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 149/162 2023-11-24T06:14:27.966 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/162 2023-11-24T06:14:27.966 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 151/162 2023-11-24T06:14:27.966 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/162 2023-11-24T06:14:27.966 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 153/162 2023-11-24T06:14:27.966 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/162 2023-11-24T06:14:27.966 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/162 2023-11-24T06:14:27.967 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/162 2023-11-24T06:14:27.967 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 157/162 2023-11-24T06:14:27.967 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 158/162 2023-11-24T06:14:27.967 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 159/162 2023-11-24T06:14:27.967 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 160/162 2023-11-24T06:14:27.967 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 161/162 2023-11-24T06:14:30.450 INFO:teuthology.orchestra.run.smithi006.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 162/162 2023-11-24T06:14:30.450 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:30.450 INFO:teuthology.orchestra.run.smithi006.stdout:Upgraded: 2023-11-24T06:14:30.450 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-24T06:14:30.450 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-24T06:14:30.451 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-24T06:14:30.451 INFO:teuthology.orchestra.run.smithi006.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-24T06:14:30.451 INFO:teuthology.orchestra.run.smithi006.stdout: acl-2.3.1-4.el9.x86_64 2023-11-24T06:14:30.451 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-24T06:14:30.451 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-24T06:14:30.451 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-24T06:14:30.451 INFO:teuthology.orchestra.run.smithi006.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-24T06:14:30.451 INFO:teuthology.orchestra.run.smithi006.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-24T06:14:30.451 INFO:teuthology.orchestra.run.smithi006.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.451 INFO:teuthology.orchestra.run.smithi006.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.451 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-24T06:14:30.451 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-24T06:14:30.451 INFO:teuthology.orchestra.run.smithi006.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-24T06:14:30.452 INFO:teuthology.orchestra.run.smithi006.stdout:Installed: 2023-11-24T06:14:30.452 INFO:teuthology.orchestra.run.smithi006.stdout: automake-1.16.2-8.el9.noarch 2023-11-24T06:14:30.452 INFO:teuthology.orchestra.run.smithi006.stdout: bison-3.7.4-5.el9.x86_64 2023-11-24T06:14:30.452 INFO:teuthology.orchestra.run.smithi006.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-24T06:14:30.452 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.452 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.452 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.452 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.452 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:30.452 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.452 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.452 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.453 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:30.453 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:30.453 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:30.453 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:30.453 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:30.453 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.453 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.453 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:30.453 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.453 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.453 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.453 INFO:teuthology.orchestra.run.smithi006.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:30.453 INFO:teuthology.orchestra.run.smithi006.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:30.454 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.454 INFO:teuthology.orchestra.run.smithi006.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:30.454 INFO:teuthology.orchestra.run.smithi006.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-24T06:14:30.454 INFO:teuthology.orchestra.run.smithi006.stdout: flex-2.6.4-9.el9.x86_64 2023-11-24T06:14:30.454 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-24T06:14:30.454 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-24T06:14:30.454 INFO:teuthology.orchestra.run.smithi006.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-24T06:14:30.454 INFO:teuthology.orchestra.run.smithi006.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-24T06:14:30.454 INFO:teuthology.orchestra.run.smithi006.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-24T06:14:30.454 INFO:teuthology.orchestra.run.smithi006.stdout: jq-1.6-16.el9.x86_64 2023-11-24T06:14:30.454 INFO:teuthology.orchestra.run.smithi006.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-24T06:14:30.455 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-24T06:14:30.455 INFO:teuthology.orchestra.run.smithi006.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-24T06:14:30.455 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.455 INFO:teuthology.orchestra.run.smithi006.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.455 INFO:teuthology.orchestra.run.smithi006.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.455 INFO:teuthology.orchestra.run.smithi006.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-24T06:14:30.455 INFO:teuthology.orchestra.run.smithi006.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-24T06:14:30.455 INFO:teuthology.orchestra.run.smithi006.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-24T06:14:30.455 INFO:teuthology.orchestra.run.smithi006.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-24T06:14:30.455 INFO:teuthology.orchestra.run.smithi006.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-24T06:14:30.455 INFO:teuthology.orchestra.run.smithi006.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.455 INFO:teuthology.orchestra.run.smithi006.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.456 INFO:teuthology.orchestra.run.smithi006.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-24T06:14:30.456 INFO:teuthology.orchestra.run.smithi006.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.456 INFO:teuthology.orchestra.run.smithi006.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-24T06:14:30.456 INFO:teuthology.orchestra.run.smithi006.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-24T06:14:30.456 INFO:teuthology.orchestra.run.smithi006.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-24T06:14:30.456 INFO:teuthology.orchestra.run.smithi006.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-24T06:14:30.456 INFO:teuthology.orchestra.run.smithi006.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-24T06:14:30.456 INFO:teuthology.orchestra.run.smithi006.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-24T06:14:30.456 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-24T06:14:30.456 INFO:teuthology.orchestra.run.smithi006.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-24T06:14:30.456 INFO:teuthology.orchestra.run.smithi006.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-24T06:14:30.456 INFO:teuthology.orchestra.run.smithi006.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-24T06:14:30.457 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-13.11-1.el9.x86_64 2023-11-24T06:14:30.457 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-24T06:14:30.457 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-24T06:14:30.457 INFO:teuthology.orchestra.run.smithi006.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-24T06:14:30.457 INFO:teuthology.orchestra.run.smithi006.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-24T06:14:30.457 INFO:teuthology.orchestra.run.smithi006.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-24T06:14:30.457 INFO:teuthology.orchestra.run.smithi006.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-24T06:14:30.457 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-24T06:14:30.457 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.457 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.457 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.457 INFO:teuthology.orchestra.run.smithi006.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-24T06:14:30.458 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-24T06:14:30.458 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-24T06:14:30.458 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-24T06:14:30.458 INFO:teuthology.orchestra.run.smithi006.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-24T06:14:30.458 INFO:teuthology.orchestra.run.smithi006.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-24T06:14:30.458 INFO:teuthology.orchestra.run.smithi006.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-24T06:14:30.458 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-24T06:14:30.458 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-24T06:14:30.458 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-24T06:14:30.458 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-24T06:14:30.458 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-24T06:14:30.458 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-24T06:14:30.459 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-24T06:14:30.459 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-24T06:14:30.459 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-24T06:14:30.459 INFO:teuthology.orchestra.run.smithi006.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-24T06:14:30.459 INFO:teuthology.orchestra.run.smithi006.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-24T06:14:30.459 INFO:teuthology.orchestra.run.smithi006.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-24T06:14:30.459 INFO:teuthology.orchestra.run.smithi006.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-24T06:14:30.459 INFO:teuthology.orchestra.run.smithi006.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-24T06:14:30.459 INFO:teuthology.orchestra.run.smithi006.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-24T06:14:30.459 INFO:teuthology.orchestra.run.smithi006.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-24T06:14:30.459 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-24T06:14:30.459 INFO:teuthology.orchestra.run.smithi006.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-24T06:14:30.459 INFO:teuthology.orchestra.run.smithi006.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-24T06:14:30.460 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-24T06:14:30.460 INFO:teuthology.orchestra.run.smithi006.stdout: python3-ply-3.11-14.el9.noarch 2023-11-24T06:14:30.460 INFO:teuthology.orchestra.run.smithi006.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-24T06:14:30.460 INFO:teuthology.orchestra.run.smithi006.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-24T06:14:30.460 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-24T06:14:30.460 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-24T06:14:30.460 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-24T06:14:30.460 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-24T06:14:30.460 INFO:teuthology.orchestra.run.smithi006.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-24T06:14:30.460 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.460 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.460 INFO:teuthology.orchestra.run.smithi006.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-24T06:14:30.460 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-24T06:14:30.461 INFO:teuthology.orchestra.run.smithi006.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-24T06:14:30.461 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.461 INFO:teuthology.orchestra.run.smithi006.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-24T06:14:30.461 INFO:teuthology.orchestra.run.smithi006.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-24T06:14:30.461 INFO:teuthology.orchestra.run.smithi006.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-24T06:14:30.461 INFO:teuthology.orchestra.run.smithi006.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-24T06:14:30.461 INFO:teuthology.orchestra.run.smithi006.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-24T06:14:30.461 INFO:teuthology.orchestra.run.smithi006.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-24T06:14:30.461 INFO:teuthology.orchestra.run.smithi006.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-24T06:14:30.461 INFO:teuthology.orchestra.run.smithi006.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-24T06:14:30.461 INFO:teuthology.orchestra.run.smithi006.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-24T06:14:30.462 INFO:teuthology.orchestra.run.smithi006.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-24T06:14:30.462 INFO:teuthology.orchestra.run.smithi006.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-24T06:14:30.462 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.462 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.462 INFO:teuthology.orchestra.run.smithi006.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:30.462 INFO:teuthology.orchestra.run.smithi006.stdout: re2-1:20211101-3.el9.x86_64 2023-11-24T06:14:30.462 INFO:teuthology.orchestra.run.smithi006.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-24T06:14:30.462 INFO:teuthology.orchestra.run.smithi006.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-24T06:14:30.462 INFO:teuthology.orchestra.run.smithi006.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-24T06:14:30.462 INFO:teuthology.orchestra.run.smithi006.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-24T06:14:30.462 INFO:teuthology.orchestra.run.smithi006.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-24T06:14:30.463 INFO:teuthology.orchestra.run.smithi006.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-24T06:14:30.463 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:30.463 INFO:teuthology.orchestra.run.smithi006.stdout:Complete! 2023-11-24T06:14:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 1/162 2023-11-24T06:14:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2/162 2023-11-24T06:14:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 3/162 2023-11-24T06:14:30.756 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 4/162 2023-11-24T06:14:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/162 2023-11-24T06:14:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 6/162 2023-11-24T06:14:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 7/162 2023-11-24T06:14:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T06:14:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T06:14:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T06:14:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 11/162 2023-11-24T06:14:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 12/162 2023-11-24T06:14:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 13/162 2023-11-24T06:14:30.757 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 14/162 2023-11-24T06:14:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 15/162 2023-11-24T06:14:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 16/162 2023-11-24T06:14:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 17/162 2023-11-24T06:14:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 18/162 2023-11-24T06:14:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 19/162 2023-11-24T06:14:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/162 2023-11-24T06:14:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 21/162 2023-11-24T06:14:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 22/162 2023-11-24T06:14:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 23/162 2023-11-24T06:14:30.758 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 24/162 2023-11-24T06:14:30.759 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 25/162 2023-11-24T06:14:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 26/162 2023-11-24T06:14:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 27/162 2023-11-24T06:14:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 28/162 2023-11-24T06:14:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/162 2023-11-24T06:14:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 30/162 2023-11-24T06:14:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 31/162 2023-11-24T06:14:30.772 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/162 2023-11-24T06:14:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/162 2023-11-24T06:14:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 34/162 2023-11-24T06:14:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/162 2023-11-24T06:14:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 36/162 2023-11-24T06:14:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 37/162 2023-11-24T06:14:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 38/162 2023-11-24T06:14:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/162 2023-11-24T06:14:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/162 2023-11-24T06:14:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/162 2023-11-24T06:14:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/162 2023-11-24T06:14:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/162 2023-11-24T06:14:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/162 2023-11-24T06:14:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/162 2023-11-24T06:14:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/162 2023-11-24T06:14:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : jq-1.6-16.el9.x86_64 47/162 2023-11-24T06:14:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/162 2023-11-24T06:14:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/162 2023-11-24T06:14:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/162 2023-11-24T06:14:30.774 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/162 2023-11-24T06:14:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/162 2023-11-24T06:14:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/162 2023-11-24T06:14:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/162 2023-11-24T06:14:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/162 2023-11-24T06:14:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/162 2023-11-24T06:14:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/162 2023-11-24T06:14:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/162 2023-11-24T06:14:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/162 2023-11-24T06:14:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/162 2023-11-24T06:14:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/162 2023-11-24T06:14:30.775 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/162 2023-11-24T06:14:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/162 2023-11-24T06:14:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/162 2023-11-24T06:14:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/162 2023-11-24T06:14:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/162 2023-11-24T06:14:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/162 2023-11-24T06:14:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/162 2023-11-24T06:14:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/162 2023-11-24T06:14:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/162 2023-11-24T06:14:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/162 2023-11-24T06:14:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/162 2023-11-24T06:14:30.776 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/162 2023-11-24T06:14:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/162 2023-11-24T06:14:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/162 2023-11-24T06:14:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/162 2023-11-24T06:14:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/162 2023-11-24T06:14:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/162 2023-11-24T06:14:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/162 2023-11-24T06:14:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/162 2023-11-24T06:14:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/162 2023-11-24T06:14:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/162 2023-11-24T06:14:30.777 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/162 2023-11-24T06:14:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/162 2023-11-24T06:14:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/162 2023-11-24T06:14:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/162 2023-11-24T06:14:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/162 2023-11-24T06:14:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/162 2023-11-24T06:14:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/162 2023-11-24T06:14:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/162 2023-11-24T06:14:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/162 2023-11-24T06:14:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/162 2023-11-24T06:14:30.778 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/162 2023-11-24T06:14:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/162 2023-11-24T06:14:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/162 2023-11-24T06:14:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/162 2023-11-24T06:14:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/162 2023-11-24T06:14:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/162 2023-11-24T06:14:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/162 2023-11-24T06:14:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/162 2023-11-24T06:14:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/162 2023-11-24T06:14:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/162 2023-11-24T06:14:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/162 2023-11-24T06:14:30.779 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/162 2023-11-24T06:14:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/162 2023-11-24T06:14:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/162 2023-11-24T06:14:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/162 2023-11-24T06:14:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/162 2023-11-24T06:14:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/162 2023-11-24T06:14:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/162 2023-11-24T06:14:30.780 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/162 2023-11-24T06:14:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/162 2023-11-24T06:14:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/162 2023-11-24T06:14:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/162 2023-11-24T06:14:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/162 2023-11-24T06:14:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/162 2023-11-24T06:14:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/162 2023-11-24T06:14:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/162 2023-11-24T06:14:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/162 2023-11-24T06:14:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/162 2023-11-24T06:14:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/162 2023-11-24T06:14:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/162 2023-11-24T06:14:30.781 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/162 2023-11-24T06:14:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/162 2023-11-24T06:14:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/162 2023-11-24T06:14:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/162 2023-11-24T06:14:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/162 2023-11-24T06:14:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/162 2023-11-24T06:14:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/162 2023-11-24T06:14:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/162 2023-11-24T06:14:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/162 2023-11-24T06:14:30.782 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/162 2023-11-24T06:14:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 133/162 2023-11-24T06:14:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/162 2023-11-24T06:14:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 135/162 2023-11-24T06:14:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/162 2023-11-24T06:14:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-11-24T06:14:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/162 2023-11-24T06:14:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/162 2023-11-24T06:14:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/162 2023-11-24T06:14:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/162 2023-11-24T06:14:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/162 2023-11-24T06:14:30.783 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/162 2023-11-24T06:14:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/162 2023-11-24T06:14:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/162 2023-11-24T06:14:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/162 2023-11-24T06:14:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/162 2023-11-24T06:14:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/162 2023-11-24T06:14:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 149/162 2023-11-24T06:14:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/162 2023-11-24T06:14:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 151/162 2023-11-24T06:14:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/162 2023-11-24T06:14:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 153/162 2023-11-24T06:14:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/162 2023-11-24T06:14:30.784 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/162 2023-11-24T06:14:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/162 2023-11-24T06:14:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 157/162 2023-11-24T06:14:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 158/162 2023-11-24T06:14:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 159/162 2023-11-24T06:14:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 160/162 2023-11-24T06:14:30.785 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 161/162 2023-11-24T06:14:31.030 DEBUG:teuthology.parallel:result is None 2023-11-24T06:14:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 162/162 2023-11-24T06:14:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:33.087 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2023-11-24T06:14:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-24T06:14:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-24T06:14:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-24T06:14:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-24T06:14:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: acl-2.3.1-4.el9.x86_64 2023-11-24T06:14:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-24T06:14:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-24T06:14:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-24T06:14:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-24T06:14:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-24T06:14:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-24T06:14:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-24T06:14:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-24T06:14:33.088 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2023-11-24T06:14:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.2-8.el9.noarch 2023-11-24T06:14:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.7.4-5.el9.x86_64 2023-11-24T06:14:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-24T06:14:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T06:14:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-24T06:14:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.4-9.el9.x86_64 2023-11-24T06:14:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-24T06:14:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-24T06:14:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-24T06:14:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-24T06:14:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-24T06:14:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: jq-1.6-16.el9.x86_64 2023-11-24T06:14:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-24T06:14:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-24T06:14:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-24T06:14:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-24T06:14:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-24T06:14:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-24T06:14:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-24T06:14:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-24T06:14:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-24T06:14:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-24T06:14:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-24T06:14:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-24T06:14:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-24T06:14:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-24T06:14:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-24T06:14:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-24T06:14:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-24T06:14:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-24T06:14:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-24T06:14:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-13.11-1.el9.x86_64 2023-11-24T06:14:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-24T06:14:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-24T06:14:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-24T06:14:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-24T06:14:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-24T06:14:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-24T06:14:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-24T06:14:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-24T06:14:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-24T06:14:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-24T06:14:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-24T06:14:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-24T06:14:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-24T06:14:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-24T06:14:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-24T06:14:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-24T06:14:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-24T06:14:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-24T06:14:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-24T06:14:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-24T06:14:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-24T06:14:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-24T06:14:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-24T06:14:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-24T06:14:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-24T06:14:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-24T06:14:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-24T06:14:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-24T06:14:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-24T06:14:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-24T06:14:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-24T06:14:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-24T06:14:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-24T06:14:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-24T06:14:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply-3.11-14.el9.noarch 2023-11-24T06:14:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-24T06:14:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-24T06:14:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-24T06:14:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-24T06:14:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-24T06:14:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-24T06:14:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-24T06:14:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-24T06:14:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-24T06:14:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-24T06:14:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-24T06:14:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-24T06:14:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-24T06:14:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-24T06:14:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-24T06:14:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-24T06:14:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-24T06:14:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-24T06:14:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-24T06:14:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-24T06:14:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-24T06:14:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T06:14:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: re2-1:20211101-3.el9.x86_64 2023-11-24T06:14:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-24T06:14:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-24T06:14:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-24T06:14:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-24T06:14:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-24T06:14:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-24T06:14:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:33.100 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2023-11-24T06:14:33.787 DEBUG:teuthology.parallel:result is None 2023-11-24T06:14:33.788 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:14:34.006 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T06:14:34.067 INFO:teuthology.orchestra.run.smithi005.stdout:18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:14:34.067 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:14:34.067 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T06:14:34.068 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T06:14:34.281 DEBUG:teuthology.orchestra.run.smithi006:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T06:14:34.344 INFO:teuthology.orchestra.run.smithi006.stdout:18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:14:34.345 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el9 2023-11-24T06:14:34.345 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T06:14:34.345 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-24T06:14:34.345 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:14:34.345 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T06:14:34.434 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:14:34.434 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T06:14:34.515 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-24T06:14:34.515 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:14:34.516 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T06:14:34.625 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T06:14:34.726 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:14:34.726 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T06:14:34.804 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T06:14:34.923 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-24T06:14:34.924 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:14:34.924 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T06:14:35.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T06:14:35.135 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:14:35.135 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T06:14:35.217 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T06:14:35.338 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-24T06:14:35.338 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:14:35.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T06:14:35.422 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T06:14:35.536 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:14:35.536 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T06:14:35.612 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T06:14:35.730 INFO:teuthology.run_tasks:Running task ceph... 2023-11-24T06:14:35.791 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-24T06:14:35.792 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /var/log/ceph 2023-11-24T06:14:35.798 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 777 /var/log/ceph 2023-11-24T06:14:35.876 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-24T06:14:35.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T06:14:35.881 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T06:14:35.969 INFO:tasks.ceph:Creating extra log directories... 2023-11-24T06:14:35.970 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T06:14:35.975 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T06:14:36.066 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-24T06:14:36.066 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-24T06:14:36.067 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-24T06:14:36.067 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:14:36.067 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T06:14:36.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T06:14:36.345 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T06:14:36.430 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:14:36.430 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T06:14:36.477 DEBUG:teuthology.orchestra.run.smithi006:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T06:14:36.692 DEBUG:teuthology.orchestra.run.smithi006:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T06:14:36.767 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-24T06:14:36.767 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-24T06:14:36.767 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552', 'branch': 'rishabh-mgr-vol-misc', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}}', 'email': None, 'first_in_suite': False, 'job_id': '7465552', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'rishabh-mgr-vol-misc'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'rishabh-mgr-vol-misc', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3305, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'rishabh-mgr-vol-misc', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi005.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHxC54AoB40hHZJBei+C+H1tj0i0mbvv+5ZiCTjBje1ZITrJ1meyYKiQ7HmT71SkrxFCLB1mDtonwt8WAVmum/Q=', 'smithi006.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIKjSSSw3SKfP6l+THmyThcZ6hDVcLI75y1IghvDKxCpc9aPaHYHE7irCADEi8Q+VWGwhXLYcR4sptlmUvccwM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-24_05:41:33', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-24T06:14:36.768 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T06:14:36.774 DEBUG:teuthology.orchestra.run.smithi006:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T06:14:36.818 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T06:14:36.822 DEBUG:teuthology.orchestra.run.smithi006:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T06:14:36.930 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:14:36.931 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T06:14:36.973 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T06:14:36.973 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2023-11-24T06:14:37.057 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T06:14:37.057 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:14:37.057 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 632 Links: 1 2023-11-24T06:14:37.057 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:14:37.057 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:14:37.057 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-11-24 06:14:26.673553496 +0000 2023-11-24T06:14:37.057 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-11-24 06:11:49.401710160 +0000 2023-11-24T06:14:37.057 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-11-24 06:11:49.401710160 +0000 2023-11-24T06:14:37.058 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2023-11-24 06:11:49.401710160 +0000 2023-11-24T06:14:37.058 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T06:14:37.170 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-11-24T06:14:37.170 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-11-24T06:14:37.170 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000255604 s, 2.0 MB/s 2023-11-24T06:14:37.173 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T06:14:37.220 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2023-11-24T06:14:37.302 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T06:14:37.302 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:14:37.302 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 651 Links: 1 2023-11-24T06:14:37.302 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:14:37.302 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:14:37.302 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-11-24 06:14:26.673553496 +0000 2023-11-24T06:14:37.302 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-11-24 06:11:50.461687734 +0000 2023-11-24T06:14:37.302 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-11-24 06:11:50.461687734 +0000 2023-11-24T06:14:37.302 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2023-11-24 06:11:50.461687734 +0000 2023-11-24T06:14:37.303 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T06:14:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-11-24T06:14:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-11-24T06:14:37.415 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000292936 s, 1.7 MB/s 2023-11-24T06:14:37.419 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T06:14:37.467 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2023-11-24T06:14:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T06:14:37.511 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:14:37.511 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 671 Links: 1 2023-11-24T06:14:37.511 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:14:37.511 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:14:37.511 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-11-24 06:14:26.675553453 +0000 2023-11-24T06:14:37.512 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-11-24 06:11:51.495665859 +0000 2023-11-24T06:14:37.512 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-11-24 06:11:51.495665859 +0000 2023-11-24T06:14:37.513 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2023-11-24 06:11:51.495665859 +0000 2023-11-24T06:14:37.513 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T06:14:37.624 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-11-24T06:14:37.624 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-11-24T06:14:37.624 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000250254 s, 2.0 MB/s 2023-11-24T06:14:37.626 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T06:14:37.671 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2023-11-24T06:14:37.758 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T06:14:37.758 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:14:37.758 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 688 Links: 1 2023-11-24T06:14:37.759 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:14:37.759 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:14:37.759 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-11-24 06:14:26.675553453 +0000 2023-11-24T06:14:37.759 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-11-24 06:11:52.527644026 +0000 2023-11-24T06:14:37.759 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-11-24 06:11:52.527644026 +0000 2023-11-24T06:14:37.759 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2023-11-24 06:11:52.527644026 +0000 2023-11-24T06:14:37.759 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T06:14:37.872 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-11-24T06:14:37.872 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-11-24T06:14:37.873 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000189307 s, 2.7 MB/s 2023-11-24T06:14:37.875 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T06:14:37.918 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:14:37.918 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:14:37.918 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T06:14:37.982 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T06:14:37.983 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_1 2023-11-24T06:14:38.037 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T06:14:38.037 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:14:38.037 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 634 Links: 1 2023-11-24T06:14:38.037 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:14:38.037 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:14:38.038 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-11-24 06:14:23.721813108 +0000 2023-11-24T06:14:38.038 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-11-24 06:11:46.888937707 +0000 2023-11-24T06:14:38.038 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-11-24 06:11:46.888937707 +0000 2023-11-24T06:14:38.038 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2023-11-24 06:11:46.888937707 +0000 2023-11-24T06:14:38.038 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T06:14:38.123 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-11-24T06:14:38.123 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-11-24T06:14:38.124 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000325494 s, 1.6 MB/s 2023-11-24T06:14:38.127 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T06:14:38.190 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_2 2023-11-24T06:14:38.247 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T06:14:38.247 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:14:38.248 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 652 Links: 1 2023-11-24T06:14:38.248 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:14:38.248 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:14:38.248 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-11-24 06:14:23.722813097 +0000 2023-11-24T06:14:38.248 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-11-24 06:11:47.951926460 +0000 2023-11-24T06:14:38.248 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-11-24 06:11:47.951926460 +0000 2023-11-24T06:14:38.248 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2023-11-24 06:11:47.951926460 +0000 2023-11-24T06:14:38.248 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T06:14:38.343 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-11-24T06:14:38.343 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-11-24T06:14:38.344 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000343718 s, 1.5 MB/s 2023-11-24T06:14:38.347 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T06:14:38.407 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_3 2023-11-24T06:14:38.461 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T06:14:38.462 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:14:38.462 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 670 Links: 1 2023-11-24T06:14:38.462 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:14:38.462 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:14:38.462 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-11-24 06:14:23.723813087 +0000 2023-11-24T06:14:38.462 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-11-24 06:11:48.998915384 +0000 2023-11-24T06:14:38.462 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-11-24 06:11:48.998915384 +0000 2023-11-24T06:14:38.462 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2023-11-24 06:11:48.998915384 +0000 2023-11-24T06:14:38.462 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T06:14:38.559 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-11-24T06:14:38.560 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-11-24T06:14:38.560 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000344998 s, 1.5 MB/s 2023-11-24T06:14:38.563 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T06:14:38.628 DEBUG:teuthology.orchestra.run.smithi006:> stat /dev/vg_nvme/lv_4 2023-11-24T06:14:38.685 INFO:teuthology.orchestra.run.smithi006.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T06:14:38.685 INFO:teuthology.orchestra.run.smithi006.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T06:14:38.685 INFO:teuthology.orchestra.run.smithi006.stdout:Device: 5h/5d Inode: 688 Links: 1 2023-11-24T06:14:38.685 INFO:teuthology.orchestra.run.smithi006.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T06:14:38.685 INFO:teuthology.orchestra.run.smithi006.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T06:14:38.685 INFO:teuthology.orchestra.run.smithi006.stdout:Access: 2023-11-24 06:14:23.723813087 +0000 2023-11-24T06:14:38.686 INFO:teuthology.orchestra.run.smithi006.stdout:Modify: 2023-11-24 06:11:50.083903905 +0000 2023-11-24T06:14:38.686 INFO:teuthology.orchestra.run.smithi006.stdout:Change: 2023-11-24 06:11:50.083903905 +0000 2023-11-24T06:14:38.687 INFO:teuthology.orchestra.run.smithi006.stdout: Birth: 2023-11-24 06:11:50.083903905 +0000 2023-11-24T06:14:38.687 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T06:14:38.785 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records in 2023-11-24T06:14:38.785 INFO:teuthology.orchestra.run.smithi006.stderr:1+0 records out 2023-11-24T06:14:38.786 INFO:teuthology.orchestra.run.smithi006.stderr:512 bytes copied, 0.000375903 s, 1.4 MB/s 2023-11-24T06:14:38.790 DEBUG:teuthology.orchestra.run.smithi006:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T06:14:38.853 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:14:38.853 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi006.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T06:14:38.853 INFO:tasks.ceph:Generating config... 2023-11-24T06:14:38.855 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-24T06:14:38.855 INFO:tasks.ceph:[client] debug client = 20 2023-11-24T06:14:38.855 INFO:tasks.ceph:[client] debug ms = 1 2023-11-24T06:14:38.855 INFO:tasks.ceph:[client] pid file = 2023-11-24T06:14:38.855 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-24T06:14:38.855 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-24T06:14:38.855 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-24T06:14:38.855 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-24T06:14:38.855 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-24T06:14:38.855 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-24T06:14:38.855 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-24T06:14:38.856 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-24T06:14:38.856 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-24T06:14:38.856 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-24T06:14:38.856 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-24T06:14:38.856 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-24T06:14:38.856 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-24T06:14:38.856 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-24T06:14:38.856 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-24T06:14:38.856 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-24T06:14:38.856 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-24T06:14:38.856 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-24T06:14:38.856 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-24T06:14:38.856 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-24T06:14:38.856 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-24T06:14:38.856 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-24T06:14:38.857 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-24T06:14:38.857 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-24T06:14:38.857 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-24T06:14:38.857 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-24T06:14:38.857 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-24T06:14:38.857 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-24T06:14:38.857 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-24T06:14:38.857 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-24T06:14:38.857 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-24T06:14:38.857 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-24T06:14:38.857 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-24T06:14:38.857 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-24T06:14:38.857 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-24T06:14:38.858 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-24T06:14:38.858 INFO:tasks.ceph:Setting up mon.a... 2023-11-24T06:14:38.858 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-24T06:14:38.981 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.keyring 2023-11-24T06:14:38.988 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-24T06:14:39.094 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:14:39.165 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.5'), ('mon.b', '172.21.15.6'), ('mon.c', '[v2:172.21.15.6:3301,v1:172.21.15.6:6790]')] 2023-11-24T06:14:39.165 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.5,172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-24T06:14:39.166 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:14:39.167 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T06:14:39.216 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.5 --add b 172.21.15.6 --addv c '[v2:172.21.15.6:3301,v1:172.21.15.6:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:14:39.375 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:14:39.375 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: generated fsid 8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c 2023-11-24T06:14:39.375 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = pacific 2023-11-24T06:14:39.375 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 0 2023-11-24T06:14:39.375 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c 2023-11-24T06:14:39.375 INFO:teuthology.orchestra.run.smithi005.stdout:last_changed 2023-11-24T06:14:39.373696+0000 2023-11-24T06:14:39.376 INFO:teuthology.orchestra.run.smithi005.stdout:created 2023-11-24T06:14:39.373696+0000 2023-11-24T06:14:39.376 INFO:teuthology.orchestra.run.smithi005.stdout:min_mon_release 16 (pacific) 2023-11-24T06:14:39.376 INFO:teuthology.orchestra.run.smithi005.stdout:election_strategy: 1 2023-11-24T06:14:39.376 INFO:teuthology.orchestra.run.smithi005.stdout:0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2023-11-24T06:14:39.376 INFO:teuthology.orchestra.run.smithi005.stdout:1: [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] mon.b 2023-11-24T06:14:39.376 INFO:teuthology.orchestra.run.smithi005.stdout:2: [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] mon.c 2023-11-24T06:14:39.376 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-24T06:14:39.379 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T06:14:39.430 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c... 2023-11-24T06:14:39.431 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T06:14:39.436 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T06:14:39.608 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2023-11-24T06:14:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2023-11-24T06:14:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T06:14:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2023-11-24T06:14:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2023-11-24T06:14:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2023-11-24T06:14:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2023-11-24T06:14:39.608 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2023-11-24T06:14:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2023-11-24T06:14:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2023-11-24T06:14:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T06:14:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2023-11-24T06:14:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T06:14:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T06:14:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2023-11-24T06:14:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.609 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2023-11-24T06:14:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2023-11-24T06:14:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2023-11-24T06:14:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2023-11-24T06:14:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2023-11-24T06:14:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2023-11-24T06:14:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2023-11-24T06:14:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T06:14:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T06:14:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2023-11-24T06:14:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T06:14:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2023-11-24T06:14:39.610 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2023-11-24T06:14:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2023-11-24T06:14:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2023-11-24T06:14:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2023-11-24T06:14:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790]" 2023-11-24T06:14:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c 2023-11-24T06:14:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.611 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2023-11-24T06:14:39.611 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug trim objects = true 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2023-11-24T06:14:39.612 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2023-11-24T06:14:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2023-11-24T06:14:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2023-11-24T06:14:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2023-11-24T06:14:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2023-11-24T06:14:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2023-11-24T06:14:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2023-11-24T06:14:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2023-11-24T06:14:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: bdev async discard = True 2023-11-24T06:14:39.613 INFO:teuthology.orchestra.run.smithi005.stdout: bdev enable discard = True 2023-11-24T06:14:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore allocator = bitmap 2023-11-24T06:14:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2023-11-24T06:14:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2023-11-24T06:14:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 1/20 2023-11-24T06:14:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 1/20 2023-11-24T06:14:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2023-11-24T06:14:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2023-11-24T06:14:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 4/10 2023-11-24T06:14:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T06:14:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2023-11-24T06:14:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2023-11-24T06:14:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2023-11-24T06:14:39.614 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2023-11-24T06:14:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2023-11-24T06:14:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.615 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2023-11-24T06:14:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2023-11-24T06:14:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2023-11-24T06:14:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2023-11-24T06:14:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2023-11-24T06:14:39.615 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:14:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:14:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2023-11-24T06:14:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2023-11-24T06:14:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: debug finisher = 20 2023-11-24T06:14:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.616 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2023-11-24T06:14:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2023-11-24T06:14:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2023-11-24T06:14:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2023-11-24T06:14:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2023-11-24T06:14:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2023-11-24T06:14:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2023-11-24T06:14:39.616 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:14:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2023-11-24T06:14:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2023-11-24T06:14:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:14:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T06:14:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T06:14:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T06:14:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T06:14:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T06:14:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T06:14:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.617 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2023-11-24T06:14:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2023-11-24T06:14:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2023-11-24T06:14:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: mon op complaint time = 120 2023-11-24T06:14:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:14:39.618 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2023-11-24T06:14:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2023-11-24T06:14:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2023-11-24T06:14:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2023-11-24T06:14:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T06:14:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T06:14:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: client mount timeout = 600 2023-11-24T06:14:39.618 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2023-11-24T06:14:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2023-11-24T06:14:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = "" 2023-11-24T06:14:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2023-11-24T06:14:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2023-11-24T06:14:39.619 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2023-11-24T06:14:39.619 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2023-11-24T06:14:39.619 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2023-11-24T06:14:39.619 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2023-11-24T06:14:39.619 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2023-11-24T06:14:39.619 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2023-11-24T06:14:39.619 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2023-11-24T06:14:39.619 INFO:teuthology.orchestra.run.smithi005.stdout:[mds] 2023-11-24T06:14:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds = 20 2023-11-24T06:14:39.619 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds balancer = 20 2023-11-24T06:14:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2023-11-24T06:14:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug frag = True 2023-11-24T06:14:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug scatterstat = True 2023-11-24T06:14:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: mds op complaint time = 180 2023-11-24T06:14:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: mds verify scatter = True 2023-11-24T06:14:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2023-11-24T06:14:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2023-11-24T06:14:39.620 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2023-11-24T06:14:39.626 INFO:teuthology.orchestra.run.smithi006.stdout:[global] 2023-11-24T06:14:39.626 INFO:teuthology.orchestra.run.smithi006.stdout: chdir = "" 2023-11-24T06:14:39.626 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T06:14:39.626 INFO:teuthology.orchestra.run.smithi006.stdout: auth supported = cephx 2023-11-24T06:14:39.626 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.626 INFO:teuthology.orchestra.run.smithi006.stdout: filestore xattr use omap = true 2023-11-24T06:14:39.626 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.626 INFO:teuthology.orchestra.run.smithi006.stdout: mon clock drift allowed = 1.000 2023-11-24T06:14:39.626 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.626 INFO:teuthology.orchestra.run.smithi006.stdout: osd crush chooseleaf type = 0 2023-11-24T06:14:39.626 INFO:teuthology.orchestra.run.smithi006.stdout: auth debug = true 2023-11-24T06:14:39.626 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.627 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on old message = true 2023-11-24T06:14:39.627 INFO:teuthology.orchestra.run.smithi006.stdout: ms die on bug = true 2023-11-24T06:14:39.627 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.627 INFO:teuthology.orchestra.run.smithi006.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T06:14:39.627 INFO:teuthology.orchestra.run.smithi006.stdout: mon pg warn max object skew = 0 2023-11-24T06:14:39.627 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.627 INFO:teuthology.orchestra.run.smithi006.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T06:14:39.627 INFO:teuthology.orchestra.run.smithi006.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T06:14:39.627 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.627 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default size = 2 2023-11-24T06:14:39.627 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.627 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow primary affinity = true 2023-11-24T06:14:39.628 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd allow pg remap = true 2023-11-24T06:14:39.628 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on legacy crush tunables = false 2023-11-24T06:14:39.628 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on crush straw calc version zero = false 2023-11-24T06:14:39.628 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on no sortbitwise = false 2023-11-24T06:14:39.628 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on osd down out interval zero = false 2023-11-24T06:14:39.628 INFO:teuthology.orchestra.run.smithi006.stdout: mon warn on too few osds = false 2023-11-24T06:14:39.628 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T06:14:39.628 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T06:14:39.628 INFO:teuthology.orchestra.run.smithi006.stdout: mon_allow_pool_size_one = true 2023-11-24T06:14:39.628 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.628 INFO:teuthology.orchestra.run.smithi006.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T06:14:39.628 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.628 INFO:teuthology.orchestra.run.smithi006.stdout: osd default data pool replay window = 5 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout: mon allow pool delete = true 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout: mon cluster log file level = debug 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout: debug asserts on shutdown = true 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout: mon health detail to clog = false 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout: mon host = "172.21.15.5,172.21.15.6,[v2:172.21.15.6:3301,v1:172.21.15.6:6790]" 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout: fsid = 8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout:[osd] 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout: osd journal size = 100 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub load threshold = 5.0 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout: osd scrub max interval = 600 2023-11-24T06:14:39.629 INFO:teuthology.orchestra.run.smithi006.stdout: osd mclock profile = high_recovery_ops 2023-11-24T06:14:39.630 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.630 INFO:teuthology.orchestra.run.smithi006.stdout: osd recover clone overlap = true 2023-11-24T06:14:39.630 INFO:teuthology.orchestra.run.smithi006.stdout: osd recovery max chunk = 1048576 2023-11-24T06:14:39.630 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.630 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug shutdown = true 2023-11-24T06:14:39.630 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug op order = true 2023-11-24T06:14:39.630 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug verify stray on activate = true 2023-11-24T06:14:39.630 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug trim objects = true 2023-11-24T06:14:39.630 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.630 INFO:teuthology.orchestra.run.smithi006.stdout: osd open classes on start = true 2023-11-24T06:14:39.630 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug pg log writeout = true 2023-11-24T06:14:39.630 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.630 INFO:teuthology.orchestra.run.smithi006.stdout: osd deep scrub update digest min age = 30 2023-11-24T06:14:39.631 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.631 INFO:teuthology.orchestra.run.smithi006.stdout: osd map max advance = 10 2023-11-24T06:14:39.631 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.631 INFO:teuthology.orchestra.run.smithi006.stdout: journal zero on create = true 2023-11-24T06:14:39.631 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.631 INFO:teuthology.orchestra.run.smithi006.stdout: filestore ondisk finisher threads = 3 2023-11-24T06:14:39.631 INFO:teuthology.orchestra.run.smithi006.stdout: filestore apply finisher threads = 3 2023-11-24T06:14:39.631 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.631 INFO:teuthology.orchestra.run.smithi006.stdout: bdev debug aio = true 2023-11-24T06:14:39.631 INFO:teuthology.orchestra.run.smithi006.stdout: osd debug misdirected ops = true 2023-11-24T06:14:39.631 INFO:teuthology.orchestra.run.smithi006.stdout: bdev async discard = True 2023-11-24T06:14:39.631 INFO:teuthology.orchestra.run.smithi006.stdout: bdev enable discard = True 2023-11-24T06:14:39.631 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore allocator = bitmap 2023-11-24T06:14:39.631 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore block size = 96636764160 2023-11-24T06:14:39.632 INFO:teuthology.orchestra.run.smithi006.stdout: bluestore fsck on mount = True 2023-11-24T06:14:39.632 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluefs = 1/20 2023-11-24T06:14:39.632 INFO:teuthology.orchestra.run.smithi006.stdout: debug bluestore = 1/20 2023-11-24T06:14:39.632 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2023-11-24T06:14:39.632 INFO:teuthology.orchestra.run.smithi006.stdout: debug osd = 20 2023-11-24T06:14:39.632 INFO:teuthology.orchestra.run.smithi006.stdout: debug rocksdb = 4/10 2023-11-24T06:14:39.632 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T06:14:39.632 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd full ratio = 0.9 2023-11-24T06:14:39.632 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd nearfull ratio = 0.8 2023-11-24T06:14:39.632 INFO:teuthology.orchestra.run.smithi006.stdout: osd failsafe full ratio = 0.95 2023-11-24T06:14:39.632 INFO:teuthology.orchestra.run.smithi006.stdout: osd objectstore = bluestore 2023-11-24T06:14:39.632 INFO:teuthology.orchestra.run.smithi006.stdout: osd op complaint time = 180 2023-11-24T06:14:39.632 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.632 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr] 2023-11-24T06:14:39.633 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2023-11-24T06:14:39.633 INFO:teuthology.orchestra.run.smithi006.stdout: debug mgr = 20 2023-11-24T06:14:39.633 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2023-11-24T06:14:39.633 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2023-11-24T06:14:39.633 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:14:39.633 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:14:39.633 INFO:teuthology.orchestra.run.smithi006.stdout: mgr/telemetry/nag = false 2023-11-24T06:14:39.633 INFO:teuthology.orchestra.run.smithi006.stdout: debug client = 20 2023-11-24T06:14:39.633 INFO:teuthology.orchestra.run.smithi006.stdout: debug finisher = 20 2023-11-24T06:14:39.633 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.633 INFO:teuthology.orchestra.run.smithi006.stdout:[mon] 2023-11-24T06:14:39.633 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2023-11-24T06:14:39.633 INFO:teuthology.orchestra.run.smithi006.stdout: debug mon = 20 2023-11-24T06:14:39.634 INFO:teuthology.orchestra.run.smithi006.stdout: debug paxos = 20 2023-11-24T06:14:39.634 INFO:teuthology.orchestra.run.smithi006.stdout: debug auth = 20 2023-11-24T06:14:39.634 INFO:teuthology.orchestra.run.smithi006.stdout: mon data avail warn = 5 2023-11-24T06:14:39.634 INFO:teuthology.orchestra.run.smithi006.stdout: mon mgr mkfs grace = 240 2023-11-24T06:14:39.634 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min pgs per osd = 4 2023-11-24T06:14:39.634 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd reporter subtree level = osd 2023-11-24T06:14:39.634 INFO:teuthology.orchestra.run.smithi006.stdout: mon osd prime pg temp = true 2023-11-24T06:14:39.634 INFO:teuthology.orchestra.run.smithi006.stdout: mon reweight min bytes per osd = 10 2023-11-24T06:14:39.634 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.634 INFO:teuthology.orchestra.run.smithi006.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T06:14:39.634 INFO:teuthology.orchestra.run.smithi006.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T06:14:39.634 INFO:teuthology.orchestra.run.smithi006.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T06:14:39.634 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.635 INFO:teuthology.orchestra.run.smithi006.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T06:14:39.635 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T06:14:39.635 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T06:14:39.635 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.635 INFO:teuthology.orchestra.run.smithi006.stdout: # 1m isn't quite enough 2023-11-24T06:14:39.635 INFO:teuthology.orchestra.run.smithi006.stdout: mon_down_mkfs_grace = 2m 2023-11-24T06:14:39.635 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.635 INFO:teuthology.orchestra.run.smithi006.stdout: mon_warn_on_filestore_osds = false 2023-11-24T06:14:39.635 INFO:teuthology.orchestra.run.smithi006.stdout: mon op complaint time = 120 2023-11-24T06:14:39.635 INFO:teuthology.orchestra.run.smithi006.stdout: 2023-11-24T06:14:39.635 INFO:teuthology.orchestra.run.smithi006.stdout:[client] 2023-11-24T06:14:39.635 INFO:teuthology.orchestra.run.smithi006.stdout: rgw cache enabled = true 2023-11-24T06:14:39.636 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable ops log = true 2023-11-24T06:14:39.636 INFO:teuthology.orchestra.run.smithi006.stdout: rgw enable usage log = true 2023-11-24T06:14:39.636 INFO:teuthology.orchestra.run.smithi006.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T06:14:39.636 INFO:teuthology.orchestra.run.smithi006.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T06:14:39.636 INFO:teuthology.orchestra.run.smithi006.stdout: client mount timeout = 600 2023-11-24T06:14:39.636 INFO:teuthology.orchestra.run.smithi006.stdout: debug client = 20 2023-11-24T06:14:39.636 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2023-11-24T06:14:39.636 INFO:teuthology.orchestra.run.smithi006.stdout: pid file = "" 2023-11-24T06:14:39.636 INFO:teuthology.orchestra.run.smithi006.stdout: rados mon op timeout = 900 2023-11-24T06:14:39.636 INFO:teuthology.orchestra.run.smithi006.stdout: rados osd op timeout = 900 2023-11-24T06:14:39.636 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.a] 2023-11-24T06:14:39.636 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.b] 2023-11-24T06:14:39.636 INFO:teuthology.orchestra.run.smithi006.stdout:[mon.c] 2023-11-24T06:14:39.636 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.a] 2023-11-24T06:14:39.637 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.c] 2023-11-24T06:14:39.637 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.b] 2023-11-24T06:14:39.637 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.d] 2023-11-24T06:14:39.637 INFO:teuthology.orchestra.run.smithi006.stdout:[mds] 2023-11-24T06:14:39.637 INFO:teuthology.orchestra.run.smithi006.stdout: debug mds = 20 2023-11-24T06:14:39.637 INFO:teuthology.orchestra.run.smithi006.stdout: debug mds balancer = 20 2023-11-24T06:14:39.637 INFO:teuthology.orchestra.run.smithi006.stdout: debug ms = 1 2023-11-24T06:14:39.637 INFO:teuthology.orchestra.run.smithi006.stdout: mds debug frag = True 2023-11-24T06:14:39.637 INFO:teuthology.orchestra.run.smithi006.stdout: mds debug scatterstat = True 2023-11-24T06:14:39.637 INFO:teuthology.orchestra.run.smithi006.stdout: mds op complaint time = 180 2023-11-24T06:14:39.637 INFO:teuthology.orchestra.run.smithi006.stdout: mds verify scatter = True 2023-11-24T06:14:39.637 INFO:teuthology.orchestra.run.smithi006.stdout: osd op complaint time = 180 2023-11-24T06:14:39.637 INFO:teuthology.orchestra.run.smithi006.stdout: rados mon op timeout = 900 2023-11-24T06:14:39.637 INFO:teuthology.orchestra.run.smithi006.stdout: rados osd op timeout = 900 2023-11-24T06:14:39.672 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-24T06:14:39.672 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-24T06:14:39.831 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-24T06:14:39.831 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:14:39.831 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-24T06:14:39.884 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:14:39.884 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-24T06:14:39.943 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T06:14:39.944 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:14:39.944 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T06:14:39.944 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:14:40.108 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:14:40.109 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:14:40.211 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T06:14:40.211 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:14:40.211 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T06:14:40.211 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T06:14:40.386 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:14:40.386 DEBUG:teuthology.orchestra.run.smithi006:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:14:40.488 INFO:tasks.ceph:Setting up mon nodes... 2023-11-24T06:14:40.488 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-24T06:14:40.489 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T06:14:40.690 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T06:14:40.700 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T06:14:40.931 INFO:teuthology.orchestra.run.smithi006.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T06:14:40.943 INFO:tasks.ceph:Setting up mds nodes... 2023-11-24T06:14:40.944 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T06:14:41.151 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T06:14:41.161 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-24T06:14:41.258 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T06:14:41.456 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T06:14:41.466 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-24T06:14:41.567 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T06:14:41.775 INFO:teuthology.orchestra.run.smithi006.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T06:14:41.785 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-24T06:14:41.887 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T06:14:42.065 INFO:teuthology.orchestra.run.smithi006.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T06:14:42.074 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-24T06:14:42.167 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-24T06:14:42.168 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-24T06:14:42.310 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-24T06:14:42.364 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-24T06:14:42.502 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-24T06:14:42.575 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-24T06:14:42.719 INFO:teuthology.orchestra.run.smithi006.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-24T06:14:42.790 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-24T06:14:42.926 INFO:teuthology.orchestra.run.smithi006.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-24T06:14:42.994 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-24T06:14:42.994 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi006.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T06:14:42.994 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-24T06:14:43.108 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:14:43.108 INFO:tasks.ceph:role: osd.0 2023-11-24T06:14:43.108 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T06:14:43.108 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T06:14:43.209 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:14:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:14:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:14:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:14:43.210 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:14:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:14:43.210 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:14:43.210 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:14:43.210 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:14:43.210 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:14:43.243 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2023-11-24T06:14:43.249 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2023-11-24T06:14:43.249 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-24T06:14:43.359 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-24T06:14:43.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-24T06:14:43.535 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:14:43.536 INFO:tasks.ceph:role: osd.1 2023-11-24T06:14:43.536 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T06:14:43.536 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T06:14:43.634 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:14:43.634 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:14:43.634 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:14:43.634 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:14:43.634 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:14:43.634 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:14:43.634 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:14:43.634 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:14:43.634 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:14:43.634 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:14:43.668 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2023-11-24T06:14:43.674 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2023-11-24T06:14:43.675 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-24T06:14:43.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-24T06:14:43.887 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-24T06:14:43.986 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:14:43.986 INFO:tasks.ceph:role: osd.2 2023-11-24T06:14:43.986 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T06:14:43.986 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T06:14:44.086 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:14:44.086 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:14:44.086 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:14:44.086 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:14:44.086 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:14:44.086 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:14:44.086 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:14:44.087 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:14:44.087 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:14:44.087 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:14:44.123 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2023-11-24T06:14:44.129 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2023-11-24T06:14:44.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-24T06:14:44.253 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-24T06:14:44.342 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-24T06:14:44.436 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T06:14:44.436 INFO:tasks.ceph:role: osd.3 2023-11-24T06:14:44.436 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T06:14:44.436 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T06:14:44.534 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:14:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:14:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:14:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:14:44.534 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:14:44.534 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:14:44.534 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:14:44.534 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:14:44.535 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:14:44.535 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:14:44.570 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2023-11-24T06:14:44.575 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2023-11-24T06:14:44.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-24T06:14:44.695 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-24T06:14:44.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:14:44.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:44.930+0000 7f25cf339500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-24T06:14:44.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:44.931+0000 7f25cf339500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-24T06:14:44.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:44.931+0000 7f25cf339500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:14:44.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:44.931+0000 7f25cf339500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:14:44.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:44.931+0000 7f25cf339500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T06:14:44.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:44.934+0000 7f25cf339500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-24T06:14:48.118 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-24T06:14:48.225 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:14:48.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:48.366+0000 7f806e213500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-24T06:14:48.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:48.367+0000 7f806e213500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-24T06:14:48.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:48.367+0000 7f806e213500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:14:48.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:48.367+0000 7f806e213500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:14:48.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:48.367+0000 7f806e213500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T06:14:48.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:48.369+0000 7f806e213500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-24T06:14:51.576 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-24T06:14:51.680 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:14:51.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:51.823+0000 7f8b73a0e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-24T06:14:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:51.824+0000 7f8b73a0e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-24T06:14:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:51.824+0000 7f8b73a0e500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:14:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:51.824+0000 7f8b73a0e500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:14:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:51.824+0000 7f8b73a0e500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T06:14:51.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:51.826+0000 7f8b73a0e500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-24T06:14:55.033 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-24T06:14:55.137 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:14:55.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:55.285+0000 7efd5ef8b500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-24T06:14:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:55.285+0000 7efd5ef8b500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-24T06:14:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:55.286+0000 7efd5ef8b500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:14:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:55.286+0000 7efd5ef8b500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:14:55.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:55.286+0000 7efd5ef8b500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T06:14:55.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:14:55.288+0000 7efd5ef8b500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-24T06:14:58.453 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-24T06:14:58.557 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-24T06:14:58.671 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:14:58.671 INFO:tasks.ceph:role: osd.4 2023-11-24T06:14:58.672 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T06:14:58.672 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T06:14:58.783 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:14:58.783 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:14:58.783 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:14:58.783 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:14:58.783 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:14:58.783 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:14:58.784 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:14:58.784 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:14:58.784 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:14:58.784 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:14:59.299 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2023-11-24T06:14:59.306 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2023-11-24T06:14:59.306 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-24T06:14:59.499 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-24T06:14:59.613 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-24T06:14:59.706 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:14:59.706 INFO:tasks.ceph:role: osd.5 2023-11-24T06:14:59.706 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T06:14:59.706 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T06:14:59.799 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:14:59.799 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:14:59.799 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:14:59.799 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:14:59.799 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:14:59.800 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:14:59.800 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:14:59.800 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:14:59.800 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:14:59.800 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:15:00.280 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2023-11-24T06:15:00.287 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2023-11-24T06:15:00.288 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-24T06:15:00.410 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-24T06:15:00.505 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-24T06:15:00.588 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:15:00.588 INFO:tasks.ceph:role: osd.6 2023-11-24T06:15:00.588 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T06:15:00.589 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T06:15:00.683 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:15:00.683 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:15:00.684 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:15:00.684 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:15:00.684 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:15:00.684 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:15:00.684 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:15:00.684 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:15:00.684 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:15:00.684 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:15:01.191 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2023-11-24T06:15:01.196 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2023-11-24T06:15:01.197 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-24T06:15:01.380 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-24T06:15:01.490 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-24T06:15:01.595 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T06:15:01.596 INFO:tasks.ceph:role: osd.7 2023-11-24T06:15:01.596 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T06:15:01.596 DEBUG:teuthology.orchestra.run.smithi006:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T06:15:01.703 INFO:teuthology.orchestra.run.smithi006.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T06:15:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T06:15:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T06:15:01.703 INFO:teuthology.orchestra.run.smithi006.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T06:15:01.703 INFO:teuthology.orchestra.run.smithi006.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T06:15:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: = sunit=0 swidth=0 blks 2023-11-24T06:15:01.704 INFO:teuthology.orchestra.run.smithi006.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T06:15:01.704 INFO:teuthology.orchestra.run.smithi006.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T06:15:01.704 INFO:teuthology.orchestra.run.smithi006.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T06:15:01.704 INFO:teuthology.orchestra.run.smithi006.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T06:15:02.175 INFO:teuthology.orchestra.run.smithi006.stdout:Discarding blocks...Done. 2023-11-24T06:15:02.182 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi006.front.sepia.ceph.com -o noatime 2023-11-24T06:15:02.182 DEBUG:teuthology.orchestra.run.smithi006:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-24T06:15:02.330 DEBUG:teuthology.orchestra.run.smithi006:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-24T06:15:02.426 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:15:02.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:02.563+0000 7f0003a1a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-24T06:15:02.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:02.569+0000 7f0003a1a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-24T06:15:02.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:02.569+0000 7f0003a1a500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:15:02.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:02.569+0000 7f0003a1a500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:15:02.572 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:02.569+0000 7f0003a1a500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T06:15:02.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:02.577+0000 7f0003a1a500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-24T06:15:05.768 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-24T06:15:05.873 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:15:06.025 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:06.022+0000 7f2de7efa500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-24T06:15:06.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:06.023+0000 7f2de7efa500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-24T06:15:06.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:06.023+0000 7f2de7efa500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:15:06.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:06.023+0000 7f2de7efa500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:15:06.026 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:06.023+0000 7f2de7efa500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T06:15:06.028 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:06.025+0000 7f2de7efa500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-24T06:15:06.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:15:06.778 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:15:08.952 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-24T06:15:09.058 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:15:09.211 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:09.208+0000 7f571a016500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-24T06:15:09.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:09.208+0000 7f571a016500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-24T06:15:09.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:09.209+0000 7f571a016500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:15:09.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:09.209+0000 7f571a016500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:15:09.212 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:09.209+0000 7f571a016500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T06:15:09.214 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:09.211+0000 7f571a016500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-24T06:15:12.389 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-24T06:15:12.496 DEBUG:teuthology.orchestra.run.smithi006:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:15:12.651 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:12.649+0000 7f71eb746500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-24T06:15:12.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:12.650+0000 7f71eb746500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-24T06:15:12.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:12.650+0000 7f71eb746500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:15:12.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:12.650+0000 7f71eb746500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:15:12.652 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:12.650+0000 7f71eb746500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T06:15:12.654 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:12.652+0000 7f71eb746500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-24T06:15:15.584 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-24T06:15:15.690 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-24T06:15:15.690 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:15:15.690 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-24T06:15:15.806 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:15:15.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-24T06:15:15.909 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:15:15.910 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-24T06:15:16.013 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:15:16.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-24T06:15:16.115 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:15:16.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-24T06:15:16.223 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:15:16.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-24T06:15:16.321 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:15:16.321 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-24T06:15:16.418 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:15:16.418 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-24T06:15:16.528 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:15:16.528 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-24T06:15:16.633 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:15:16.633 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-24T06:15:16.736 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:15:16.736 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-24T06:15:16.836 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:15:16.836 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-24T06:15:16.936 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:15:16.936 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-24T06:15:17.042 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:15:17.042 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-24T06:15:17.147 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:15:17.147 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-24T06:15:17.208 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:15:17.209 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-24T06:15:17.267 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:15:17.267 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-24T06:15:17.329 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:15:17.330 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-24T06:15:17.389 INFO:tasks.ceph:Adding keys to all mons... 2023-11-24T06:15:17.389 DEBUG:teuthology.orchestra.run.smithi005:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T06:15:17.398 DEBUG:teuthology.orchestra.run.smithi006:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T06:15:17.489 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr.y] 2023-11-24T06:15:17.489 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBQP2BlifcJKRAAtFRCDbxtaYpJsjkEzsiflg== 2023-11-24T06:15:17.489 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.a] 2023-11-24T06:15:17.489 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBRP2BlyCrzCBAAJesPTcst5+wrepefjCUvHg== 2023-11-24T06:15:17.489 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.c] 2023-11-24T06:15:17.490 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBRP2BlLkkdGxAAEyqzUDjb03E28fqgPW7Pyw== 2023-11-24T06:15:17.490 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.0] 2023-11-24T06:15:17.490 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBUP2BlkY6CNxAAO6xMBc0Nn3XUzwMGBQSpCA== 2023-11-24T06:15:17.490 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.1] 2023-11-24T06:15:17.490 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBYP2BlDfTdFRAA2Uzod+xA1fIi7o5hDEjPlA== 2023-11-24T06:15:17.490 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.2] 2023-11-24T06:15:17.490 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBbP2BlipkbMRAAWHdPwet86lhQ8vw7+KmMSw== 2023-11-24T06:15:17.490 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.3] 2023-11-24T06:15:17.490 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBfP2BlcVoWERAA1k6C6OGIqowWtwirtn5plA== 2023-11-24T06:15:17.490 INFO:teuthology.orchestra.run.smithi006.stdout:[mgr.x] 2023-11-24T06:15:17.490 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBQP2BlG8xzNxAA2ttxOCQMN3FLEsaYKkjPwA== 2023-11-24T06:15:17.490 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.b] 2023-11-24T06:15:17.490 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBRP2BlVMklLhAAYbNoUFBrX220YLRYI/XfaQ== 2023-11-24T06:15:17.491 INFO:teuthology.orchestra.run.smithi006.stdout:[mds.d] 2023-11-24T06:15:17.491 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBSP2Blm77TAxAA0HmGIgVqVEw/cTWRusU5BQ== 2023-11-24T06:15:17.491 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.4] 2023-11-24T06:15:17.491 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBmP2BldD2nIRAAWSGnSO7f+h3lMUpR2djc7g== 2023-11-24T06:15:17.491 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.5] 2023-11-24T06:15:17.491 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBqP2Bll0JtARAAdsNKrm3d7LhvMb8gaLkqFg== 2023-11-24T06:15:17.491 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.6] 2023-11-24T06:15:17.491 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBtP2Blgr5+DBAARFEkq6Vxl4H3pckjHhnB8A== 2023-11-24T06:15:17.491 INFO:teuthology.orchestra.run.smithi006.stdout:[osd.7] 2023-11-24T06:15:17.491 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBwP2Bl+VrAJhAAJY4lbiL0VUVC4Psg8at2uQ== 2023-11-24T06:15:17.491 INFO:teuthology.orchestra.run.smithi006.stdout:[client.0] 2023-11-24T06:15:17.491 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBSP2Bl7ShkEhAAJ7REPm4GEzxnc5jqGBmDlQ== 2023-11-24T06:15:17.492 INFO:teuthology.orchestra.run.smithi006.stdout:[client.2] 2023-11-24T06:15:17.492 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBSP2Bl7UbUHRAAGxHoa/vos5RXgzS1+fswPQ== 2023-11-24T06:15:17.492 INFO:teuthology.orchestra.run.smithi006.stdout:[client.1] 2023-11-24T06:15:17.492 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBSP2BlvArOKhAAWiNFDPr+pkxXGvpS22wZwQ== 2023-11-24T06:15:17.492 INFO:teuthology.orchestra.run.smithi006.stdout:[client.3] 2023-11-24T06:15:17.492 INFO:teuthology.orchestra.run.smithi006.stdout: key = AQBSP2Bl/eUYNxAAMHVblVlSugTel+jhnGiEUA== 2023-11-24T06:15:17.492 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.y] 2023-11-24T06:15:17.492 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBQP2BlifcJKRAAtFRCDbxtaYpJsjkEzsiflg== 2023-11-24T06:15:17.492 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2023-11-24T06:15:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBRP2BlyCrzCBAAJesPTcst5+wrepefjCUvHg== 2023-11-24T06:15:17.493 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c] 2023-11-24T06:15:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBRP2BlLkkdGxAAEyqzUDjb03E28fqgPW7Pyw== 2023-11-24T06:15:17.493 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.0] 2023-11-24T06:15:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBUP2BlkY6CNxAAO6xMBc0Nn3XUzwMGBQSpCA== 2023-11-24T06:15:17.493 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.1] 2023-11-24T06:15:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBYP2BlDfTdFRAA2Uzod+xA1fIi7o5hDEjPlA== 2023-11-24T06:15:17.493 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.2] 2023-11-24T06:15:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBbP2BlipkbMRAAWHdPwet86lhQ8vw7+KmMSw== 2023-11-24T06:15:17.493 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.3] 2023-11-24T06:15:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBfP2BlcVoWERAA1k6C6OGIqowWtwirtn5plA== 2023-11-24T06:15:17.493 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.x] 2023-11-24T06:15:17.493 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBQP2BlG8xzNxAA2ttxOCQMN3FLEsaYKkjPwA== 2023-11-24T06:15:17.494 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2023-11-24T06:15:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBRP2BlVMklLhAAYbNoUFBrX220YLRYI/XfaQ== 2023-11-24T06:15:17.494 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d] 2023-11-24T06:15:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBSP2Blm77TAxAA0HmGIgVqVEw/cTWRusU5BQ== 2023-11-24T06:15:17.494 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.4] 2023-11-24T06:15:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBmP2BldD2nIRAAWSGnSO7f+h3lMUpR2djc7g== 2023-11-24T06:15:17.494 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.5] 2023-11-24T06:15:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBqP2Bll0JtARAAdsNKrm3d7LhvMb8gaLkqFg== 2023-11-24T06:15:17.494 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.6] 2023-11-24T06:15:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBtP2Blgr5+DBAARFEkq6Vxl4H3pckjHhnB8A== 2023-11-24T06:15:17.494 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.7] 2023-11-24T06:15:17.494 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBwP2Bl+VrAJhAAJY4lbiL0VUVC4Psg8at2uQ== 2023-11-24T06:15:17.495 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2023-11-24T06:15:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBSP2Bl7ShkEhAAJ7REPm4GEzxnc5jqGBmDlQ== 2023-11-24T06:15:17.495 INFO:teuthology.orchestra.run.smithi005.stdout:[client.2] 2023-11-24T06:15:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBSP2Bl7UbUHRAAGxHoa/vos5RXgzS1+fswPQ== 2023-11-24T06:15:17.495 INFO:teuthology.orchestra.run.smithi005.stdout:[client.1] 2023-11-24T06:15:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBSP2BlvArOKhAAWiNFDPr+pkxXGvpS22wZwQ== 2023-11-24T06:15:17.495 INFO:teuthology.orchestra.run.smithi005.stdout:[client.3] 2023-11-24T06:15:17.495 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBSP2Bl/eUYNxAAMHVblVlSugTel+jhnGiEUA== 2023-11-24T06:15:17.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:15:17.505 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:15:17.660 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:15:17.669 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:15:17.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:15:17.830 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:15:17.983 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:17.992 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:18.148 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:18.154 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:18.309 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:18.314 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:18.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:18.471 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:18.626 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:15:18.635 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T06:15:18.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:15:18.799 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:15:18.953 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:15:18.960 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T06:15:19.111 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:19.120 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:19.280 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:19.290 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:19.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:19.457 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:19.613 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:19.621 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T06:15:19.779 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:15:19.788 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:15:19.944 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:15:19.949 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:15:20.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:15:20.114 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:15:20.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:15:20.283 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T06:15:20.439 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-24T06:15:20.440 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-24T06:15:20.541 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:15:20.707 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-24T06:15:20.802 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-24T06:15:20.903 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:15:21.067 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-24T06:15:21.160 DEBUG:teuthology.orchestra.run.smithi006:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-24T06:15:21.262 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T06:15:21.423 DEBUG:teuthology.orchestra.run.smithi006:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-24T06:15:21.520 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:15:21.529 DEBUG:teuthology.orchestra.run.smithi006:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T06:15:21.585 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-24T06:15:21.586 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-24T06:15:21.586 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-24T06:15:21.591 INFO:tasks.ceph.mon.a:Started 2023-11-24T06:15:21.592 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-24T06:15:21.592 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-24T06:15:21.597 INFO:tasks.ceph.mon.b:Started 2023-11-24T06:15:21.597 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-24T06:15:21.598 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-24T06:15:21.606 INFO:tasks.ceph.mon.c:Started 2023-11-24T06:15:21.606 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-24T06:15:21.606 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-24T06:15:21.606 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-24T06:15:21.614 INFO:tasks.ceph.mgr.y:Started 2023-11-24T06:15:21.615 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-24T06:15:21.615 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-24T06:15:21.620 INFO:tasks.ceph.mgr.x:Started 2023-11-24T06:15:21.620 INFO:tasks.ceph:Setting crush tunables to default 2023-11-24T06:15:21.620 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph osd crush tunables default 2023-11-24T06:15:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.802+0000 7f91a2256640 1 Processor -- start 2023-11-24T06:15:21.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.802+0000 7f91a2256640 1 -- start start 2023-11-24T06:15:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.803+0000 7f91a2256640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 0x7f919c0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.803+0000 7f91a2256640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 0x7f919c0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.803+0000 7f91a2256640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f919c15b730 con 0x7f919c09cb30 2023-11-24T06:15:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.803+0000 7f91a2256640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f919c0ac740 con 0x7f919c15b350 2023-11-24T06:15:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.803+0000 7f91a2256640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919c0ac460 con 0x7f919c0adb60 2023-11-24T06:15:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.804+0000 7f9192ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f919c09cb30 0x7f919c0b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44146/0 (socket says 172.21.15.5:44146) 2023-11-24T06:15:21.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.804+0000 7f9192ffd640 1 -- 172.21.15.5:0/523218048 learned_addr learned my addr 172.21.15.5:0/523218048 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.804+0000 7f919bfff640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 0x7f919c0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.804+0000 7f919b7fe640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 0x7f919c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.804+0000 7f919bfff640 1 -- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 msgr2=0x7f919c0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-24T06:15:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.804+0000 7f919bfff640 1 -- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 msgr2=0x7f919c0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:15:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.804+0000 7f919bfff640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 0x7f919c0adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:15:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.804+0000 7f919bfff640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 0x7f919c0adfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:15:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.804+0000 7f919b7fe640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 msgr2=0x7f919c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:15:21.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.804+0000 7f919b7fe640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 msgr2=0x7f919c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:15:21.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.804+0000 7f919b7fe640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 0x7f919c0abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:15:21.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:21.804+0000 7f919b7fe640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 0x7f919c0abe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:15:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.005+0000 7f919bfff640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 0x7f919c0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.005+0000 7f919b7fe640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 0x7f919c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.006+0000 7f919bfff640 1 -- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 msgr2=0x7f919c0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:15:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.006+0000 7f919bfff640 1 -- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 msgr2=0x7f919c0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:15:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.006+0000 7f919bfff640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 0x7f919c0adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:15:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.006+0000 7f919bfff640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 0x7f919c0adfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T06:15:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.006+0000 7f919b7fe640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 msgr2=0x7f919c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-24T06:15:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.006+0000 7f919b7fe640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 msgr2=0x7f919c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:15:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.006+0000 7f919b7fe640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 0x7f919c0abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:15:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.006+0000 7f919b7fe640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 0x7f919c0abe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T06:15:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.407+0000 7f919bfff640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 0x7f919c0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.407+0000 7f919b7fe640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 0x7f919c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.407+0000 7f919bfff640 1 -- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 msgr2=0x7f919c0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:15:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.407+0000 7f919bfff640 1 -- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 msgr2=0x7f919c0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:15:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.407+0000 7f919bfff640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 0x7f919c0adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:15:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.408+0000 7f919b7fe640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 msgr2=0x7f919c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-24T06:15:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.408+0000 7f919b7fe640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 msgr2=0x7f919c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:15:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.408+0000 7f919b7fe640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 0x7f919c0abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:15:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.408+0000 7f919bfff640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 0x7f919c0adfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T06:15:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:22.408+0000 7f919b7fe640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 0x7f919c0abe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T06:15:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:23.209+0000 7f919bfff640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 0x7f919c0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:23.209+0000 7f919b7fe640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 0x7f919c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:23.210+0000 7f919bfff640 1 -- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 msgr2=0x7f919c0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T06:15:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:23.210+0000 7f919bfff640 1 -- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 msgr2=0x7f919c0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:15:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:23.210+0000 7f919bfff640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 0x7f919c0adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:15:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:23.210+0000 7f919bfff640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 0x7f919c0adfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-24T06:15:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:23.210+0000 7f919b7fe640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 msgr2=0x7f919c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-24T06:15:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:23.210+0000 7f919b7fe640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 msgr2=0x7f919c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:15:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:23.210+0000 7f919b7fe640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 0x7f919c0abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:15:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:23.210+0000 7f919b7fe640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 0x7f919c0abe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-24T06:15:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:24.804+0000 7f9199ffb640 1 -- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 msgr2=0x7f919c0adfb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:15:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:24.804+0000 7f9199ffb640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f919c0adb60 0x7f919c0adfb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:24.804+0000 7f9199ffb640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 msgr2=0x7f919c0abe80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:15:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:24.804+0000 7f9199ffb640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.6:3300/0 conn(0x7f919c15b350 0x7f919c0abe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:24.804+0000 7f9199ffb640 1 -- 172.21.15.5:0/523218048 >> v1:172.21.15.5:6789/0 conn(0x7f919c09cb30 legacy=0x7f919c0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:24.804+0000 7f9199ffb640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 0x7f917c009810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:24.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:24.804+0000 7f9199ffb640 1 -- 172.21.15.5:0/523218048 --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f919c0b2a50 con 0x7f917c0036a0 2023-11-24T06:15:24.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:24.804+0000 7f9199ffb640 1 -- 172.21.15.5:0/523218048 --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f919c0ac740 con 0x7f917c007350 2023-11-24T06:15:24.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:24.804+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 0x7f917c009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:24.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:24.805+0000 7f9192ffd640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 msgr2=0x7f917c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:15:24.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:24.805+0000 7f9192ffd640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 msgr2=0x7f917c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:15:24.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:24.805+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 0x7f917c009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:15:24.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:24.805+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 0x7f917c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T06:15:25.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:25.006+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 0x7f917c009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:25.007+0000 7f9192ffd640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 msgr2=0x7f917c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:15:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:25.007+0000 7f9192ffd640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 msgr2=0x7f917c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:15:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:25.007+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 0x7f917c009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:15:25.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:25.007+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 0x7f917c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T06:15:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:25.408+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 0x7f917c009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:25.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:25.408+0000 7f9192ffd640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 msgr2=0x7f917c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:15:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:25.408+0000 7f9192ffd640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 msgr2=0x7f917c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:15:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:25.408+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 0x7f917c009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:15:25.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:25.408+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 0x7f917c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-24T06:15:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:26.210+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 0x7f917c009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:26.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:26.210+0000 7f9192ffd640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 msgr2=0x7f917c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-24T06:15:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:26.210+0000 7f9192ffd640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 msgr2=0x7f917c009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T06:15:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:26.210+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 0x7f917c009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T06:15:26.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:26.211+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 0x7f917c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-24T06:15:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.071+0000 7f919affd640 1 -- 172.21.15.5:0/523218048 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386042849 0 0) 0x7f919c0b2a50 con 0x7f917c0036a0 2023-11-24T06:15:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.071+0000 7f919affd640 1 -- 172.21.15.5:0/523218048 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9188003520 con 0x7f917c0036a0 2023-11-24T06:15:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.078+0000 7f919affd640 1 -- 172.21.15.5:0/523218048 <== mon.1 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2820724928 0 0) 0x7f9188003520 con 0x7f917c0036a0 2023-11-24T06:15:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.078+0000 7f919affd640 1 -- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 msgr2=0x7f917c009810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T06:15:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.078+0000 7f919affd640 1 --2- 172.21.15.5:0/523218048 >> v2:172.21.15.5:3300/0 conn(0x7f917c007350 0x7f917c009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.078+0000 7f919affd640 1 -- 172.21.15.5:0/523218048 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c00af10 con 0x7f917c0036a0 2023-11-24T06:15:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.078+0000 7f919affd640 1 -- 172.21.15.5:0/523218048 <== mon.1 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f918c00d580 con 0x7f917c0036a0 2023-11-24T06:15:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.078+0000 7f919affd640 1 -- 172.21.15.5:0/523218048 >> v1:172.21.15.6:6789/0 conn(0x7f917c0036a0 legacy=0x7f917c005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.078+0000 7f919bfff640 1 -- 172.21.15.5:0/523218048 reap_dead start 2023-11-24T06:15:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.078+0000 7f919affd640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91880039c0 0x7f9188003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.079+0000 7f919affd640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91880047c0 0x7f9188004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.079+0000 7f919affd640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9188005380 0x7f918800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.079+0000 7f919affd640 1 -- 172.21.15.5:0/523218048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c0ac740 con 0x7f91880039c0 2023-11-24T06:15:27.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.079+0000 7f919affd640 1 -- 172.21.15.5:0/523218048 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918800c240 con 0x7f9188005380 2023-11-24T06:15:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.079+0000 7f919affd640 1 -- 172.21.15.5:0/523218048 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918800c400 con 0x7f91880047c0 2023-11-24T06:15:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.079+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91880047c0 0x7f9188004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.079+0000 7f919b7fe640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91880039c0 0x7f9188003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.080+0000 7f9192ffd640 1 -- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9188005380 msgr2=0x7f918800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.080+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9188005380 0x7f918800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:27.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.080+0000 7f9192ffd640 1 -- 172.21.15.5:0/523218048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91880039c0 msgr2=0x7f9188003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:27.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.080+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91880039c0 0x7f9188003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:27.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.080+0000 7f9192ffd640 1 -- 172.21.15.5:0/523218048 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9188003520 con 0x7f91880047c0 2023-11-24T06:15:27.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.082+0000 7f9192ffd640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91880047c0 0x7f9188004c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9184007d20 tx=0x7f91880057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac24eece28e32f50 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:27.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.085+0000 7f919affd640 1 -- 172.21.15.5:0/523218048 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918402ce70 con 0x7f91880047c0 2023-11-24T06:15:27.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.085+0000 7f919affd640 1 -- 172.21.15.5:0/523218048 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9184030070 con 0x7f91880047c0 2023-11-24T06:15:27.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.085+0000 7f919affd640 1 -- 172.21.15.5:0/523218048 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9184036620 con 0x7f91880047c0 2023-11-24T06:15:27.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.086+0000 7f91a2256640 1 -- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91880047c0 msgr2=0x7f9188004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:27.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.086+0000 7f91a2256640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91880047c0 0x7f9188004c40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9184007d20 tx=0x7f91880057f0 comp rx=0 tx=0).stop 2023-11-24T06:15:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.087+0000 7f91a2256640 1 -- 172.21.15.5:0/523218048 shutdown_connections 2023-11-24T06:15:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.087+0000 7f91a2256640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91880047c0 0x7f9188004c40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.087+0000 7f91a2256640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9188005380 0x7f918800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.087+0000 7f91a2256640 1 --2- 172.21.15.5:0/523218048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91880039c0 0x7f9188003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.087+0000 7f91a2256640 1 -- 172.21.15.5:0/523218048 >> 172.21.15.5:0/523218048 conn(0x7f919c0b0ba0 msgr2=0x7f919c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.087+0000 7f91a2256640 1 -- 172.21.15.5:0/523218048 shutdown_connections 2023-11-24T06:15:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.087+0000 7f91a2256640 1 -- 172.21.15.5:0/523218048 wait complete. 2023-11-24T06:15:27.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.090+0000 7f91a2256640 1 Processor -- start 2023-11-24T06:15:27.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.090+0000 7f91a2256640 1 -- start start 2023-11-24T06:15:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.092+0000 7f91a2256640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91880039c0 0x7f919c14e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.092+0000 7f91a2256640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91880047c0 0x7f919c13df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.092+0000 7f91a2256640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9188005380 0x7f919c153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.092+0000 7f91a2256640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c13e990 con 0x7f9188005380 2023-11-24T06:15:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.092+0000 7f91a2256640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919c13ec70 con 0x7f91880039c0 2023-11-24T06:15:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.092+0000 7f91a2256640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919c13ef50 con 0x7f91880047c0 2023-11-24T06:15:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.092+0000 7f919b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91880039c0 0x7f919c14e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:27.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.092+0000 7f919bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9188005380 0x7f919c153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.092+0000 7f919b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91880039c0 0x7f919c14e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:37318/0 (socket says 172.21.15.5:37318) 2023-11-24T06:15:27.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.092+0000 7f919b7fe640 1 -- 172.21.15.5:0/2536700339 learned_addr learned my addr 172.21.15.5:0/2536700339 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.093+0000 7f9192ffd640 1 --2- 172.21.15.5:0/2536700339 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91880047c0 0x7f919c13df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.093+0000 7f919b7fe640 1 -- 172.21.15.5:0/2536700339 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91880047c0 msgr2=0x7f919c13df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.093+0000 7f919b7fe640 1 --2- 172.21.15.5:0/2536700339 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91880047c0 0x7f919c13df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.093+0000 7f919b7fe640 1 -- 172.21.15.5:0/2536700339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9188005380 msgr2=0x7f919c153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.093+0000 7f919b7fe640 1 --2- 172.21.15.5:0/2536700339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9188005380 0x7f919c153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:27.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.093+0000 7f919b7fe640 1 -- 172.21.15.5:0/2536700339 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919c13f230 con 0x7f91880039c0 2023-11-24T06:15:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.093+0000 7f919bfff640 1 --2- 172.21.15.5:0/2536700339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9188005380 0x7f919c153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:15:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.093+0000 7f919b7fe640 1 --2- 172.21.15.5:0/2536700339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91880039c0 0x7f919c14e090 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f918c00d750 tx=0x7f918c016e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.094+0000 7f9198ff9640 1 -- 172.21.15.5:0/2536700339 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c014990 con 0x7f91880039c0 2023-11-24T06:15:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.094+0000 7f9198ff9640 1 -- 172.21.15.5:0/2536700339 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918c01b070 con 0x7f91880039c0 2023-11-24T06:15:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.094+0000 7f91a2256640 1 -- 172.21.15.5:0/2536700339 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919c13f510 con 0x7f91880039c0 2023-11-24T06:15:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.094+0000 7f9198ff9640 1 -- 172.21.15.5:0/2536700339 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918c014d70 con 0x7f91880039c0 2023-11-24T06:15:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.094+0000 7f91a2256640 1 -- 172.21.15.5:0/2536700339 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f919c152fa0 con 0x7f91880039c0 2023-11-24T06:15:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.094+0000 7f9198ff9640 1 -- 172.21.15.5:0/2536700339 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f918c01d030 con 0x7f91880039c0 2023-11-24T06:15:27.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.094+0000 7f9198ff9640 1 -- 172.21.15.5:0/2536700339 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f918c017780 con 0x7f91880039c0 2023-11-24T06:15:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.095+0000 7f91a2256640 1 -- 172.21.15.5:0/2536700339 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f919c13f6a0 con 0x7f91880039c0 2023-11-24T06:15:27.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.099+0000 7f9198ff9640 1 -- 172.21.15.5:0/2536700339 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f918c017940 con 0x7f91880039c0 2023-11-24T06:15:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.173+0000 7f91a2256640 1 -- 172.21.15.5:0/2536700339 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f919c06e6d0 con 0x7f91880039c0 2023-11-24T06:15:27.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.182+0000 7f9198ff9640 1 -- 172.21.15.5:0/2536700339 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f918c017ae0 con 0x7f91880039c0 2023-11-24T06:15:27.185 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2023-11-24T06:15:27.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.184+0000 7f91a2256640 1 -- 172.21.15.5:0/2536700339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91880039c0 msgr2=0x7f919c14e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:27.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.184+0000 7f91a2256640 1 --2- 172.21.15.5:0/2536700339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91880039c0 0x7f919c14e090 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f918c00d750 tx=0x7f918c016e80 comp rx=0 tx=0).stop 2023-11-24T06:15:27.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.184+0000 7f91a2256640 1 -- 172.21.15.5:0/2536700339 shutdown_connections 2023-11-24T06:15:27.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.184+0000 7f91a2256640 1 --2- 172.21.15.5:0/2536700339 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f91880047c0 0x7f919c13df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:27.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.184+0000 7f91a2256640 1 --2- 172.21.15.5:0/2536700339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9188005380 0x7f919c153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:27.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.184+0000 7f91a2256640 1 --2- 172.21.15.5:0/2536700339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91880039c0 0x7f919c14e090 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:27.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.184+0000 7f91a2256640 1 -- 172.21.15.5:0/2536700339 >> 172.21.15.5:0/2536700339 conn(0x7f919c0b0ba0 msgr2=0x7f919c071610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:27.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.184+0000 7f91a2256640 1 -- 172.21.15.5:0/2536700339 shutdown_connections 2023-11-24T06:15:27.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:27.184+0000 7f91a2256640 1 -- 172.21.15.5:0/2536700339 wait complete. 2023-11-24T06:15:27.205 INFO:tasks.ceph:check_enable_crimson: False 2023-11-24T06:15:27.205 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-24T06:15:27.205 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:15:27.205 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-24T06:15:27.295 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:15:27.295 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-24T06:15:27.429 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:15:27.429 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-24T06:15:27.511 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:15:27.512 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-24T06:15:27.644 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:15:27.644 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-24T06:15:27.648 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:27.646+0000 7f3a1d62e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T06:15:27.710 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:27.708+0000 7f3a1d62e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T06:15:27.736 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:27.734+0000 7f1d2dc3b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T06:15:27.742 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:15:27.742 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-24T06:15:27.804 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:27.803+0000 7f1d2dc3b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T06:15:27.834 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:15:27.834 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-24T06:15:27.921 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:15:27.921 DEBUG:teuthology.orchestra.run.smithi006:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-24T06:15:28.068 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new 657f6d5f-230f-4b17-8906-54b4d9352abd 0 2023-11-24T06:15:28.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.259+0000 7f486882c640 1 Processor -- start 2023-11-24T06:15:28.261 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.259+0000 7f486882c640 1 -- start start 2023-11-24T06:15:28.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.260+0000 7f486882c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4864144770 0x7f486415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.260+0000 7f486882c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f486415f0a0 0x7f48640ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.260+0000 7f486882c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486415f930 con 0x7f48640b35a0 2023-11-24T06:15:28.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.261+0000 7f486882c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48640b43b0 con 0x7f486415f0a0 2023-11-24T06:15:28.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.261+0000 7f486882c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486415f530 con 0x7f4864144770 2023-11-24T06:15:28.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.261+0000 7f4862d9d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4864144770 0x7f486415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.261+0000 7f4862d9d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4864144770 0x7f486415eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:58872/0 (socket says 172.21.15.6:58872) 2023-11-24T06:15:28.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.261+0000 7f4862d9d640 1 -- 172.21.15.6:0/3374621911 learned_addr learned my addr 172.21.15.6:0/3374621911 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-24T06:15:28.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.261+0000 7f486359e640 1 --2- 172.21.15.6:0/3374621911 >> v2:172.21.15.5:3300/0 conn(0x7f486415f0a0 0x7f48640ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.262+0000 7f4862d9d640 1 -- 172.21.15.6:0/3374621911 >> v2:172.21.15.5:3300/0 conn(0x7f486415f0a0 msgr2=0x7f48640ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:28.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.262+0000 7f4862d9d640 1 --2- 172.21.15.6:0/3374621911 >> v2:172.21.15.5:3300/0 conn(0x7f486415f0a0 0x7f48640ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.262+0000 7f4862d9d640 1 -- 172.21.15.6:0/3374621911 >> v1:172.21.15.5:6789/0 conn(0x7f48640b35a0 legacy=0x7f486406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.262+0000 7f4862d9d640 1 -- 172.21.15.6:0/3374621911 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48640ce9f0 con 0x7f4864144770 2023-11-24T06:15:28.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.263+0000 7f4862d9d640 1 --2- 172.21.15.6:0/3374621911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4864144770 0x7f486415eb60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f48540098a0 tx=0x7f485402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84451867177815fe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:28.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.263+0000 7f486259c640 1 -- 172.21.15.6:0/3374621911 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854030840 con 0x7f4864144770 2023-11-24T06:15:28.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.263+0000 7f486259c640 1 -- 172.21.15.6:0/3374621911 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48540309a0 con 0x7f4864144770 2023-11-24T06:15:28.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.263+0000 7f486259c640 1 -- 172.21.15.6:0/3374621911 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854030d30 con 0x7f4864144770 2023-11-24T06:15:28.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.264+0000 7f486882c640 1 -- 172.21.15.6:0/3374621911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4864144770 msgr2=0x7f486415eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.264+0000 7f486882c640 1 --2- 172.21.15.6:0/3374621911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4864144770 0x7f486415eb60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f48540098a0 tx=0x7f485402b980 comp rx=0 tx=0).stop 2023-11-24T06:15:28.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.264+0000 7f486882c640 1 -- 172.21.15.6:0/3374621911 shutdown_connections 2023-11-24T06:15:28.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.264+0000 7f486882c640 1 --2- 172.21.15.6:0/3374621911 >> v2:172.21.15.5:3300/0 conn(0x7f486415f0a0 0x7f48640ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.264+0000 7f486882c640 1 --2- 172.21.15.6:0/3374621911 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4864144770 0x7f486415eb60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.264+0000 7f486882c640 1 -- 172.21.15.6:0/3374621911 >> 172.21.15.6:0/3374621911 conn(0x7f48640cec10 msgr2=0x7f48640cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:28.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.264+0000 7f486882c640 1 -- 172.21.15.6:0/3374621911 shutdown_connections 2023-11-24T06:15:28.267 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.264+0000 7f486882c640 1 -- 172.21.15.6:0/3374621911 wait complete. 2023-11-24T06:15:28.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.266+0000 7f486882c640 1 Processor -- start 2023-11-24T06:15:28.268 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.266+0000 7f486882c640 1 -- start start 2023-11-24T06:15:28.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.267+0000 7f486882c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48640b35a0 0x7f486413a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.270 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.267+0000 7f486882c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4864144770 0x7f486414ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.267+0000 7f486882c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f486415f0a0 0x7f486414cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.267+0000 7f486882c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486414bb50 con 0x7f48640b35a0 2023-11-24T06:15:28.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.267+0000 7f486882c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486414be30 con 0x7f4864144770 2023-11-24T06:15:28.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.267+0000 7f486882c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486414c110 con 0x7f486415f0a0 2023-11-24T06:15:28.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.267+0000 7f486359e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f486415f0a0 0x7f486414cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.267+0000 7f485bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4864144770 0x7f486414ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.267+0000 7f486359e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f486415f0a0 0x7f486414cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:58886/0 (socket says 172.21.15.6:58886) 2023-11-24T06:15:28.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.267+0000 7f486359e640 1 -- 172.21.15.6:0/946693142 learned_addr learned my addr 172.21.15.6:0/946693142 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-24T06:15:28.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.267+0000 7f4862d9d640 1 --2- 172.21.15.6:0/946693142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48640b35a0 0x7f486413a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.271 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.267+0000 7f486359e640 1 -- 172.21.15.6:0/946693142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4864144770 msgr2=0x7f486414ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.267+0000 7f486359e640 1 --2- 172.21.15.6:0/946693142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4864144770 0x7f486414ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.268+0000 7f486359e640 1 -- 172.21.15.6:0/946693142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48640b35a0 msgr2=0x7f486413a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.268+0000 7f486359e640 1 --2- 172.21.15.6:0/946693142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48640b35a0 0x7f486413a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.268+0000 7f486359e640 1 -- 172.21.15.6:0/946693142 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48540094d0 con 0x7f486415f0a0 2023-11-24T06:15:28.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.268+0000 7f4862d9d640 1 --2- 172.21.15.6:0/946693142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48640b35a0 0x7f486413a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:15:28.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.268+0000 7f486359e640 1 --2- 172.21.15.6:0/946693142 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f486415f0a0 0x7f486414cbf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f485c00efc0 tx=0x7f485c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:28.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.268+0000 7f485b7fe640 1 -- 172.21.15.6:0/946693142 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c014480 con 0x7f486415f0a0 2023-11-24T06:15:28.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.268+0000 7f485b7fe640 1 -- 172.21.15.6:0/946693142 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485c00f040 con 0x7f486415f0a0 2023-11-24T06:15:28.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.268+0000 7f485b7fe640 1 -- 172.21.15.6:0/946693142 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c014820 con 0x7f486415f0a0 2023-11-24T06:15:28.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.268+0000 7f486882c640 1 -- 172.21.15.6:0/946693142 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f486414c3f0 con 0x7f486415f0a0 2023-11-24T06:15:28.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.268+0000 7f486882c640 1 -- 172.21.15.6:0/946693142 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f486414b770 con 0x7f486415f0a0 2023-11-24T06:15:28.272 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.269+0000 7f485b7fe640 1 -- 172.21.15.6:0/946693142 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f485c014c80 con 0x7f486415f0a0 2023-11-24T06:15:28.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.269+0000 7f485b7fe640 1 -- 172.21.15.6:0/946693142 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f485c01de20 con 0x7f486415f0a0 2023-11-24T06:15:28.273 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.269+0000 7f486882c640 1 -- 172.21.15.6:0/946693142 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f486414c910 con 0x7f486415f0a0 2023-11-24T06:15:28.274 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.272+0000 7f485b7fe640 1 -- 172.21.15.6:0/946693142 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f486414c910 con 0x7f486415f0a0 2023-11-24T06:15:28.348 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:28.346+0000 7f3a1d62e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T06:15:28.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.351+0000 7f486882c640 1 -- 172.21.15.6:0/946693142 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd new", "uuid": "657f6d5f-230f-4b17-8906-54b4d9352abd", "id": 0} v 0) v1 -- 0x7f486404e840 con 0x7f486415f0a0 2023-11-24T06:15:28.361 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.359+0000 7f485b7fe640 1 -- 172.21.15.6:0/946693142 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "657f6d5f-230f-4b17-8906-54b4d9352abd", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f485c018020 con 0x7f486415f0a0 2023-11-24T06:15:28.361 INFO:teuthology.orchestra.run.smithi006.stdout:0 2023-11-24T06:15:28.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.361+0000 7f486882c640 1 -- 172.21.15.6:0/946693142 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f486415f0a0 msgr2=0x7f486414cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.361+0000 7f486882c640 1 --2- 172.21.15.6:0/946693142 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f486415f0a0 0x7f486414cbf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f485c00efc0 tx=0x7f485c00c490 comp rx=0 tx=0).stop 2023-11-24T06:15:28.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.361+0000 7f486882c640 1 -- 172.21.15.6:0/946693142 shutdown_connections 2023-11-24T06:15:28.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.361+0000 7f486882c640 1 --2- 172.21.15.6:0/946693142 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f486415f0a0 0x7f486414cbf0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.361+0000 7f486882c640 1 --2- 172.21.15.6:0/946693142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4864144770 0x7f486414ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.361+0000 7f486882c640 1 --2- 172.21.15.6:0/946693142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48640b35a0 0x7f486413a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.363 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.361+0000 7f486882c640 1 -- 172.21.15.6:0/946693142 >> 172.21.15.6:0/946693142 conn(0x7f48640cec10 msgr2=0x7f4864070130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:28.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.361+0000 7f486882c640 1 -- 172.21.15.6:0/946693142 shutdown_connections 2023-11-24T06:15:28.364 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.361+0000 7f486882c640 1 -- 172.21.15.6:0/946693142 wait complete. 2023-11-24T06:15:28.380 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new 00ada985-d8e1-4b65-bbb6-e7a0d651c82c 1 2023-11-24T06:15:28.414 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:28.412+0000 7f3a1d62e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T06:15:28.467 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:28.465+0000 7f3a1d62e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T06:15:28.533 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:28.531+0000 7f3a1d62e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T06:15:28.548 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:28.547+0000 7f1d2dc3b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T06:15:28.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.573+0000 7f94caccf640 1 Processor -- start 2023-11-24T06:15:28.575 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.574+0000 7f94caccf640 1 -- start start 2023-11-24T06:15:28.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.575+0000 7f94caccf640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f94c415b7a0 0x7f94c40d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.575+0000 7f94caccf640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c4144f80 con 0x7f94c415ee40 2023-11-24T06:15:28.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.575+0000 7f94caccf640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c40cfb00 con 0x7f94c40b2b30 2023-11-24T06:15:28.577 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.575+0000 7f94caccf640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c415c6d0 con 0x7f94c415b7a0 2023-11-24T06:15:28.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.575+0000 7f94c8a6b640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f94c40b2b30 0x7f94c40b2f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:40470/0 (socket says 172.21.15.6:40470) 2023-11-24T06:15:28.578 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.575+0000 7f94c8a6b640 1 -- 172.21.15.6:0/699947175 learned_addr learned my addr 172.21.15.6:0/699947175 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-24T06:15:28.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.576+0000 7f94c926c640 1 --2- 172.21.15.6:0/699947175 >> v2:172.21.15.5:3300/0 conn(0x7f94c415b7a0 0x7f94c40d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.576+0000 7f94c37fe640 1 -- 172.21.15.6:0/699947175 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1154920831 0 0) 0x7f94c40cfb00 con 0x7f94c40b2b30 2023-11-24T06:15:28.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.576+0000 7f94c37fe640 1 -- 172.21.15.6:0/699947175 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94b4003520 con 0x7f94c40b2b30 2023-11-24T06:15:28.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.576+0000 7f94c37fe640 1 -- 172.21.15.6:0/699947175 <== mon.1 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2570076647 0 0) 0x7f94b4003520 con 0x7f94c40b2b30 2023-11-24T06:15:28.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.576+0000 7f94c37fe640 1 -- 172.21.15.6:0/699947175 >> v2:172.21.15.5:3300/0 conn(0x7f94c415b7a0 msgr2=0x7f94c40d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:28.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.576+0000 7f94c37fe640 1 --2- 172.21.15.6:0/699947175 >> v2:172.21.15.5:3300/0 conn(0x7f94c415b7a0 0x7f94c40d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.576+0000 7f94c37fe640 1 -- 172.21.15.6:0/699947175 >> v1:172.21.15.5:6789/0 conn(0x7f94c415ee40 legacy=0x7f94c40b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.576+0000 7f94c37fe640 1 -- 172.21.15.6:0/699947175 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c415c110 con 0x7f94c40b2b30 2023-11-24T06:15:28.580 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.576+0000 7f94c37fe640 1 -- 172.21.15.6:0/699947175 <== mon.1 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f94b8001670 con 0x7f94c40b2b30 2023-11-24T06:15:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.576+0000 7f94c37fe640 1 -- 172.21.15.6:0/699947175 >> v1:172.21.15.6:6789/0 conn(0x7f94c40b2b30 legacy=0x7f94c40b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c37fe640 1 --2- 172.21.15.6:0/699947175 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94b4003970 0x7f94b4003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c37fe640 1 --2- 172.21.15.6:0/699947175 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94b4004370 0x7f94b4004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c37fe640 1 --2- 172.21.15.6:0/699947175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94b40050e0 0x7f94b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c37fe640 1 -- 172.21.15.6:0/699947175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c415c6d0 con 0x7f94b40050e0 2023-11-24T06:15:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c37fe640 1 -- 172.21.15.6:0/699947175 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b400c020 con 0x7f94b4003970 2023-11-24T06:15:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c37fe640 1 -- 172.21.15.6:0/699947175 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b400c1e0 con 0x7f94b4004370 2023-11-24T06:15:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c8a6b640 1 --2- 172.21.15.6:0/699947175 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94b4003970 0x7f94b4003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c3fff640 1 --2- 172.21.15.6:0/699947175 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94b4004370 0x7f94b4004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c926c640 1 --2- 172.21.15.6:0/699947175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94b40050e0 0x7f94b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c8a6b640 1 -- 172.21.15.6:0/699947175 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94b4004370 msgr2=0x7f94b4004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:28.581 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c8a6b640 1 --2- 172.21.15.6:0/699947175 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94b4004370 0x7f94b4004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c8a6b640 1 -- 172.21.15.6:0/699947175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94b40050e0 msgr2=0x7f94b400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:28.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c8a6b640 1 --2- 172.21.15.6:0/699947175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94b40050e0 0x7f94b400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c8a6b640 1 -- 172.21.15.6:0/699947175 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b4003520 con 0x7f94b4003970 2023-11-24T06:15:28.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c926c640 1 --2- 172.21.15.6:0/699947175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94b40050e0 0x7f94b400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:15:28.582 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.577+0000 7f94c926c640 1 -- 172.21.15.6:0/699947175 reap_dead start 2023-11-24T06:15:28.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.579+0000 7f94c8a6b640 1 --2- 172.21.15.6:0/699947175 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94b4003970 0x7f94b4003d50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94b800c190 tx=0x7f94b40057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e26f5fb9f2315285 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:28.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.579+0000 7f94c37fe640 1 -- 172.21.15.6:0/699947175 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8005ad0 con 0x7f94b4003970 2023-11-24T06:15:28.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.579+0000 7f94c37fe640 1 -- 172.21.15.6:0/699947175 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b8005c50 con 0x7f94b4003970 2023-11-24T06:15:28.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.579+0000 7f94c37fe640 1 -- 172.21.15.6:0/699947175 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b802c830 con 0x7f94b4003970 2023-11-24T06:15:28.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.580+0000 7f94caccf640 1 -- 172.21.15.6:0/699947175 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94b4003970 msgr2=0x7f94b4003d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.583 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.580+0000 7f94caccf640 1 --2- 172.21.15.6:0/699947175 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94b4003970 0x7f94b4003d50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94b800c190 tx=0x7f94b40057f0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.580+0000 7f94caccf640 1 -- 172.21.15.6:0/699947175 shutdown_connections 2023-11-24T06:15:28.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.580+0000 7f94caccf640 1 --2- 172.21.15.6:0/699947175 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94b4003970 0x7f94b4003d50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.580+0000 7f94caccf640 1 -- 172.21.15.6:0/699947175 >> 172.21.15.6:0/699947175 conn(0x7f94c40b63c0 msgr2=0x7f94c40b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:28.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.580+0000 7f94caccf640 1 -- 172.21.15.6:0/699947175 shutdown_connections 2023-11-24T06:15:28.584 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.580+0000 7f94caccf640 1 -- 172.21.15.6:0/699947175 wait complete. 2023-11-24T06:15:28.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.582+0000 7f94caccf640 1 Processor -- start 2023-11-24T06:15:28.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.582+0000 7f94caccf640 1 -- start start 2023-11-24T06:15:28.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.583+0000 7f94caccf640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94b4003970 0x7f94c415a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.583+0000 7f94caccf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c41558d0 0x7f94c4155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.583+0000 7f94caccf640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94c41414c0 0x7f94c4141970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.586 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.583+0000 7f94caccf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c41f4290 con 0x7f94c41558d0 2023-11-24T06:15:28.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.583+0000 7f94caccf640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c41f33d0 con 0x7f94c41414c0 2023-11-24T06:15:28.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.583+0000 7f94caccf640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4156290 con 0x7f94b4003970 2023-11-24T06:15:28.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.583+0000 7f94c8a6b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94b4003970 0x7f94c415a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.583+0000 7f94c8a6b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94b4003970 0x7f94c415a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:58900/0 (socket says 172.21.15.6:58900) 2023-11-24T06:15:28.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.583+0000 7f94c8a6b640 1 -- 172.21.15.6:0/229574159 learned_addr learned my addr 172.21.15.6:0/229574159 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-24T06:15:28.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.583+0000 7f94c926c640 1 --2- 172.21.15.6:0/229574159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94c41414c0 0x7f94c4141970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.583+0000 7f94c3fff640 1 --2- 172.21.15.6:0/229574159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c41558d0 0x7f94c4155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.584+0000 7f94c8a6b640 1 -- 172.21.15.6:0/229574159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94c41414c0 msgr2=0x7f94c4141970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.584+0000 7f94c8a6b640 1 --2- 172.21.15.6:0/229574159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94c41414c0 0x7f94c4141970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.584+0000 7f94c8a6b640 1 -- 172.21.15.6:0/229574159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c41558d0 msgr2=0x7f94c4155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.584+0000 7f94c8a6b640 1 --2- 172.21.15.6:0/229574159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c41558d0 0x7f94c4155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.587 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.584+0000 7f94c8a6b640 1 -- 172.21.15.6:0/229574159 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c41f3610 con 0x7f94b4003970 2023-11-24T06:15:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.584+0000 7f94c3fff640 1 --2- 172.21.15.6:0/229574159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c41558d0 0x7f94c4155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:15:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.584+0000 7f94c8a6b640 1 --2- 172.21.15.6:0/229574159 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94b4003970 0x7f94c415a310 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f94b800c190 tx=0x7f94b802e070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.584+0000 7f94c17fa640 1 -- 172.21.15.6:0/229574159 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b803c040 con 0x7f94b4003970 2023-11-24T06:15:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.584+0000 7f94c17fa640 1 -- 172.21.15.6:0/229574159 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b802cc40 con 0x7f94b4003970 2023-11-24T06:15:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.584+0000 7f94c17fa640 1 -- 172.21.15.6:0/229574159 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8007070 con 0x7f94b4003970 2023-11-24T06:15:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.584+0000 7f94caccf640 1 -- 172.21.15.6:0/229574159 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c41f38f0 con 0x7f94b4003970 2023-11-24T06:15:28.588 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.584+0000 7f94caccf640 1 -- 172.21.15.6:0/229574159 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c41f3eb0 con 0x7f94b4003970 2023-11-24T06:15:28.589 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:28.583+0000 7f3a1d62e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T06:15:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.585+0000 7f94c17fa640 1 -- 172.21.15.6:0/229574159 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f94b8032070 con 0x7f94b4003970 2023-11-24T06:15:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.585+0000 7f94c17fa640 1 -- 172.21.15.6:0/229574159 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f94b8035d10 con 0x7f94b4003970 2023-11-24T06:15:28.589 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.586+0000 7f94caccf640 1 -- 172.21.15.6:0/229574159 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c40b2630 con 0x7f94b4003970 2023-11-24T06:15:28.591 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.590+0000 7f94c17fa640 1 -- 172.21.15.6:0/229574159 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f94b804f060 con 0x7f94b4003970 2023-11-24T06:15:28.618 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:28.616+0000 7f1d2dc3b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T06:15:28.669 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.668+0000 7f94caccf640 1 -- 172.21.15.6:0/229574159 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00ada985-d8e1-4b65-bbb6-e7a0d651c82c", "id": 1} v 0) v1 -- 0x7f94c404e920 con 0x7f94b4003970 2023-11-24T06:15:28.671 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:28.670+0000 7f1d2dc3b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T06:15:28.675 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.674+0000 7f94c17fa640 1 -- 172.21.15.6:0/229574159 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00ada985-d8e1-4b65-bbb6-e7a0d651c82c", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f94b8007c80 con 0x7f94b4003970 2023-11-24T06:15:28.676 INFO:teuthology.orchestra.run.smithi006.stdout:1 2023-11-24T06:15:28.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.675+0000 7f94caccf640 1 -- 172.21.15.6:0/229574159 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94b4003970 msgr2=0x7f94c415a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.675+0000 7f94caccf640 1 --2- 172.21.15.6:0/229574159 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94b4003970 0x7f94c415a310 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f94b800c190 tx=0x7f94b802e070 comp rx=0 tx=0).stop 2023-11-24T06:15:28.677 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.675+0000 7f94caccf640 1 -- 172.21.15.6:0/229574159 shutdown_connections 2023-11-24T06:15:28.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.675+0000 7f94caccf640 1 --2- 172.21.15.6:0/229574159 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f94c41414c0 0x7f94c4141970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.675+0000 7f94caccf640 1 --2- 172.21.15.6:0/229574159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c41558d0 0x7f94c4155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.675+0000 7f94caccf640 1 --2- 172.21.15.6:0/229574159 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f94b4003970 0x7f94c415a310 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.675+0000 7f94caccf640 1 -- 172.21.15.6:0/229574159 >> 172.21.15.6:0/229574159 conn(0x7f94c40b63c0 msgr2=0x7f94c40d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:28.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.675+0000 7f94caccf640 1 -- 172.21.15.6:0/229574159 shutdown_connections 2023-11-24T06:15:28.678 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.675+0000 7f94caccf640 1 -- 172.21.15.6:0/229574159 wait complete. 2023-11-24T06:15:28.695 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf 2 2023-11-24T06:15:28.705 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:28.703+0000 7f3a1d62e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T06:15:28.742 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:28.740+0000 7f1d2dc3b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T06:15:28.789 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:28.787+0000 7f3a1d62e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T06:15:28.795 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:28.793+0000 7f1d2dc3b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T06:15:28.844 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:28.842+0000 7f3a1d62e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T06:15:28.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.890+0000 7fe1e1367640 1 Processor -- start 2023-11-24T06:15:28.892 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.890+0000 7fe1e1367640 1 -- start start 2023-11-24T06:15:28.893 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.891+0000 7fe1e1367640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe1dc0c8930 0x7fe1dc0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.891+0000 7fe1e1367640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1dc0c6bf0 con 0x7fe1dc0d1d90 2023-11-24T06:15:28.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.891+0000 7fe1e1367640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1dc0cedc0 con 0x7fe1dc15d3b0 2023-11-24T06:15:28.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.891+0000 7fe1e1367640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1dc0c6e40 con 0x7fe1dc0c8930 2023-11-24T06:15:28.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.892+0000 7fe1dbfff640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fe1dc15d3b0 0x7fe1dc0d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:40472/0 (socket says 172.21.15.6:40472) 2023-11-24T06:15:28.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.892+0000 7fe1dbfff640 1 -- 172.21.15.6:0/2088108814 learned_addr learned my addr 172.21.15.6:0/2088108814 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-24T06:15:28.894 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.892+0000 7fe1e0b66640 1 --2- 172.21.15.6:0/2088108814 >> v2:172.21.15.6:3300/0 conn(0x7fe1dc0c8930 0x7fe1dc0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.892+0000 7fe1daffd640 1 -- 172.21.15.6:0/2088108814 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4293520136 0 0) 0x7fe1dc0cedc0 con 0x7fe1dc15d3b0 2023-11-24T06:15:28.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.892+0000 7fe1daffd640 1 -- 172.21.15.6:0/2088108814 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1c4003520 con 0x7fe1dc15d3b0 2023-11-24T06:15:28.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.892+0000 7fe1e0b66640 1 -- 172.21.15.6:0/2088108814 >> v1:172.21.15.6:6789/0 conn(0x7fe1dc15d3b0 legacy=0x7fe1dc0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.892+0000 7fe1e0b66640 1 -- 172.21.15.6:0/2088108814 >> v1:172.21.15.5:6789/0 conn(0x7fe1dc0d1d90 legacy=0x7fe1dc0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.895 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.892+0000 7fe1e0b66640 1 -- 172.21.15.6:0/2088108814 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0c7120 con 0x7fe1dc0c8930 2023-11-24T06:15:28.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.894+0000 7fe1e0b66640 1 --2- 172.21.15.6:0/2088108814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1dc0c8930 0x7fe1dc0ce6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009d90 tx=0x7fe1d402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55c64d9dc50ba841 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:28.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.894+0000 7fe1daffd640 1 -- 172.21.15.6:0/2088108814 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d402cc60 con 0x7fe1dc0c8930 2023-11-24T06:15:28.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.894+0000 7fe1daffd640 1 -- 172.21.15.6:0/2088108814 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d402cdc0 con 0x7fe1dc0c8930 2023-11-24T06:15:28.896 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.894+0000 7fe1daffd640 1 -- 172.21.15.6:0/2088108814 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4034660 con 0x7fe1dc0c8930 2023-11-24T06:15:28.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.895+0000 7fe1e1367640 1 -- 172.21.15.6:0/2088108814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1dc0c8930 msgr2=0x7fe1dc0ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.895+0000 7fe1e1367640 1 --2- 172.21.15.6:0/2088108814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1dc0c8930 0x7fe1dc0ce6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4009d90 tx=0x7fe1d402beb0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.897 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.895+0000 7fe1e1367640 1 -- 172.21.15.6:0/2088108814 shutdown_connections 2023-11-24T06:15:28.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.895+0000 7fe1e1367640 1 --2- 172.21.15.6:0/2088108814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1dc0c8930 0x7fe1dc0ce6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.895+0000 7fe1e1367640 1 -- 172.21.15.6:0/2088108814 >> 172.21.15.6:0/2088108814 conn(0x7fe1dc0d30a0 msgr2=0x7fe1dc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:28.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.895+0000 7fe1e1367640 1 -- 172.21.15.6:0/2088108814 shutdown_connections 2023-11-24T06:15:28.898 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.895+0000 7fe1e1367640 1 -- 172.21.15.6:0/2088108814 wait complete. 2023-11-24T06:15:28.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.897+0000 7fe1e1367640 1 Processor -- start 2023-11-24T06:15:28.899 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.897+0000 7fe1e1367640 1 -- start start 2023-11-24T06:15:28.900 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.898+0000 7fe1e1367640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1dc0c8930 0x7fe1dc159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.898+0000 7fe1e1367640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1dc0d1d90 0x7fe1dc157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.898+0000 7fe1e1367640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1dc15d3b0 0x7fe1dc0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:28.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.898+0000 7fe1e1367640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0b6cb0 con 0x7fe1dc0d1d90 2023-11-24T06:15:28.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.898+0000 7fe1e1367640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0b7830 con 0x7fe1dc0c8930 2023-11-24T06:15:28.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.898+0000 7fe1e1367640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc0b6f00 con 0x7fe1dc15d3b0 2023-11-24T06:15:28.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.898+0000 7fe1dbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1dc0c8930 0x7fe1dc159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.898+0000 7fe1dbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1dc0c8930 0x7fe1dc159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40362/0 (socket says 172.21.15.6:40362) 2023-11-24T06:15:28.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.898+0000 7fe1dbfff640 1 -- 172.21.15.6:0/1818229053 learned_addr learned my addr 172.21.15.6:0/1818229053 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-24T06:15:28.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.898+0000 7fe1e0b66640 1 --2- 172.21.15.6:0/1818229053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1dc15d3b0 0x7fe1dc0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.898+0000 7fe1db7fe640 1 --2- 172.21.15.6:0/1818229053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1dc0d1d90 0x7fe1dc157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:28.901 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.899+0000 7fe1dbfff640 1 -- 172.21.15.6:0/1818229053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1dc15d3b0 msgr2=0x7fe1dc0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.899+0000 7fe1dbfff640 1 --2- 172.21.15.6:0/1818229053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1dc15d3b0 0x7fe1dc0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.899+0000 7fe1dbfff640 1 -- 172.21.15.6:0/1818229053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1dc0d1d90 msgr2=0x7fe1dc157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.899+0000 7fe1dbfff640 1 --2- 172.21.15.6:0/1818229053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1dc0d1d90 0x7fe1dc157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.899+0000 7fe1dbfff640 1 -- 172.21.15.6:0/1818229053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d4009a70 con 0x7fe1dc0c8930 2023-11-24T06:15:28.902 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.899+0000 7fe1dbfff640 1 --2- 172.21.15.6:0/1818229053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1dc0c8930 0x7fe1dc159370 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00c410 tx=0x7fe1cc00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.899+0000 7fe1d8ff9640 1 -- 172.21.15.6:0/1818229053 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc005ce0 con 0x7fe1dc0c8930 2023-11-24T06:15:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.899+0000 7fe1d8ff9640 1 -- 172.21.15.6:0/1818229053 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc005e40 con 0x7fe1dc0c8930 2023-11-24T06:15:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.899+0000 7fe1e1367640 1 -- 172.21.15.6:0/1818229053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1dc0b71e0 con 0x7fe1dc0c8930 2023-11-24T06:15:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.899+0000 7fe1d8ff9640 1 -- 172.21.15.6:0/1818229053 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc0072f0 con 0x7fe1dc0c8930 2023-11-24T06:15:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.899+0000 7fe1e1367640 1 -- 172.21.15.6:0/1818229053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1dc157880 con 0x7fe1dc0c8930 2023-11-24T06:15:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.900+0000 7fe1d8ff9640 1 -- 172.21.15.6:0/1818229053 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe1cc01c440 con 0x7fe1dc0c8930 2023-11-24T06:15:28.903 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.900+0000 7fe1d8ff9640 1 -- 172.21.15.6:0/1818229053 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fe1cc0128c0 con 0x7fe1dc0c8930 2023-11-24T06:15:28.904 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.900+0000 7fe1e1367640 1 -- 172.21.15.6:0/1818229053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1dc04e840 con 0x7fe1dc0c8930 2023-11-24T06:15:28.905 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.903+0000 7fe1d8ff9640 1 -- 172.21.15.6:0/1818229053 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fe1cc017070 con 0x7fe1dc0c8930 2023-11-24T06:15:28.907 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:28.905+0000 7f3a1d62e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T06:15:28.919 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:28.917+0000 7f1d2dc3b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T06:15:28.965 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:28.963+0000 7f3a1d62e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T06:15:28.983 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.981+0000 7fe1e1367640 1 -- 172.21.15.6:0/1818229053 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf", "id": 2} v 0) v1 -- 0x7fe1dc0ce970 con 0x7fe1dc0c8930 2023-11-24T06:15:28.989 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.987+0000 7fe1d8ff9640 1 -- 172.21.15.6:0/1818229053 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe1cc0125b0 con 0x7fe1dc0c8930 2023-11-24T06:15:28.989 INFO:teuthology.orchestra.run.smithi006.stdout:2 2023-11-24T06:15:28.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.988+0000 7fe1e1367640 1 -- 172.21.15.6:0/1818229053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1dc0c8930 msgr2=0x7fe1dc159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:28.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.989+0000 7fe1e1367640 1 --2- 172.21.15.6:0/1818229053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1dc0c8930 0x7fe1dc159370 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00c410 tx=0x7fe1cc00c8d0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.989+0000 7fe1e1367640 1 -- 172.21.15.6:0/1818229053 shutdown_connections 2023-11-24T06:15:28.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.989+0000 7fe1e1367640 1 --2- 172.21.15.6:0/1818229053 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe1dc15d3b0 0x7fe1dc0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.989+0000 7fe1e1367640 1 --2- 172.21.15.6:0/1818229053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1dc0d1d90 0x7fe1dc157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.989+0000 7fe1e1367640 1 --2- 172.21.15.6:0/1818229053 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe1dc0c8930 0x7fe1dc159370 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:28.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.989+0000 7fe1e1367640 1 -- 172.21.15.6:0/1818229053 >> 172.21.15.6:0/1818229053 conn(0x7fe1dc0d30a0 msgr2=0x7fe1dc0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:28.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.989+0000 7fe1e1367640 1 -- 172.21.15.6:0/1818229053 shutdown_connections 2023-11-24T06:15:28.991 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:28.989+0000 7fe1e1367640 1 -- 172.21.15.6:0/1818229053 wait complete. 2023-11-24T06:15:29.005 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:29.003+0000 7f1d2dc3b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T06:15:29.008 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new 388d7df5-5c6c-45db-832b-a306b901fe33 3 2023-11-24T06:15:29.061 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:29.059+0000 7f1d2dc3b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T06:15:29.126 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:29.124+0000 7f1d2dc3b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T06:15:29.185 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:29.183+0000 7f1d2dc3b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T06:15:29.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.252+0000 7fb34a9ba640 1 Processor -- start 2023-11-24T06:15:29.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.252+0000 7fb34a9ba640 1 -- start start 2023-11-24T06:15:29.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.254+0000 7fb34a9ba640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb34406e3d0 0x7fb344070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:29.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.254+0000 7fb34a9ba640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34406ae10 con 0x7fb344071310 2023-11-24T06:15:29.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.254+0000 7fb34a9ba640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34415e3b0 con 0x7fb34415df80 2023-11-24T06:15:29.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.254+0000 7fb34a9ba640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3440717c0 con 0x7fb34406e3d0 2023-11-24T06:15:29.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.254+0000 7fb3499b8640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb34406e3d0 0x7fb344070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:29.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.254+0000 7fb34a1b9640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fb34415df80 0x7fb3440b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:40474/0 (socket says 172.21.15.6:40474) 2023-11-24T06:15:29.256 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.254+0000 7fb34a1b9640 1 -- 172.21.15.6:0/3856673044 learned_addr learned my addr 172.21.15.6:0/3856673044 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-24T06:15:29.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.255+0000 7fb3499b8640 1 -- 172.21.15.6:0/3856673044 >> v1:172.21.15.6:6789/0 conn(0x7fb34415df80 legacy=0x7fb3440b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.255+0000 7fb3499b8640 1 -- 172.21.15.6:0/3856673044 >> v1:172.21.15.5:6789/0 conn(0x7fb344071310 legacy=0x7fb34415d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.255+0000 7fb3489b6640 1 -- 172.21.15.6:0/3856673044 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419317970 0 0) 0x7fb34415e3b0 con 0x7fb34415df80 2023-11-24T06:15:29.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.255+0000 7fb3499b8640 1 -- 172.21.15.6:0/3856673044 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34406d9e0 con 0x7fb34406e3d0 2023-11-24T06:15:29.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.256+0000 7fb3499b8640 1 --2- 172.21.15.6:0/3856673044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb34406e3d0 0x7fb344070dd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb34406dbd0 tx=0x7fb34002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73eeec63c6094012 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:29.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.256+0000 7fb3489b6640 1 -- 172.21.15.6:0/3856673044 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340030610 con 0x7fb34406e3d0 2023-11-24T06:15:29.258 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.256+0000 7fb3489b6640 1 -- 172.21.15.6:0/3856673044 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb340030770 con 0x7fb34406e3d0 2023-11-24T06:15:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.256+0000 7fb3489b6640 1 -- 172.21.15.6:0/3856673044 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340030b00 con 0x7fb34406e3d0 2023-11-24T06:15:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.257+0000 7fb34a9ba640 1 -- 172.21.15.6:0/3856673044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb34406e3d0 msgr2=0x7fb344070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.257+0000 7fb34a9ba640 1 --2- 172.21.15.6:0/3856673044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb34406e3d0 0x7fb344070dd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb34406dbd0 tx=0x7fb34002b960 comp rx=0 tx=0).stop 2023-11-24T06:15:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.257+0000 7fb34a9ba640 1 -- 172.21.15.6:0/3856673044 shutdown_connections 2023-11-24T06:15:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.257+0000 7fb34a9ba640 1 --2- 172.21.15.6:0/3856673044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb34406e3d0 0x7fb344070dd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.259 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.257+0000 7fb34a9ba640 1 -- 172.21.15.6:0/3856673044 >> 172.21.15.6:0/3856673044 conn(0x7fb3440b15c0 msgr2=0x7fb3440b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.257+0000 7fb34a9ba640 1 -- 172.21.15.6:0/3856673044 shutdown_connections 2023-11-24T06:15:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.257+0000 7fb34a9ba640 1 -- 172.21.15.6:0/3856673044 wait complete. 2023-11-24T06:15:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.259+0000 7fb34a9ba640 1 Processor -- start 2023-11-24T06:15:29.260 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.259+0000 7fb34a9ba640 1 -- start start 2023-11-24T06:15:29.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb34a9ba640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb34406e3d0 0x7fb34415f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:29.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb34a9ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb344071310 0x7fb344158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:29.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb34a9ba640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb34415df80 0x7fb3440affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:29.262 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb34a9ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440af240 con 0x7fb344071310 2023-11-24T06:15:29.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb34a9ba640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440af4e0 con 0x7fb34406e3d0 2023-11-24T06:15:29.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb34a9ba640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440af7c0 con 0x7fb34415df80 2023-11-24T06:15:29.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb3499b8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb34406e3d0 0x7fb34415f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:29.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb3499b8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb34406e3d0 0x7fb34415f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40384/0 (socket says 172.21.15.6:40384) 2023-11-24T06:15:29.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb3499b8640 1 -- 172.21.15.6:0/2215248577 learned_addr learned my addr 172.21.15.6:0/2215248577 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-24T06:15:29.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb34a1b9640 1 --2- 172.21.15.6:0/2215248577 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb34415df80 0x7fb3440affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:29.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb3491b7640 1 --2- 172.21.15.6:0/2215248577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb344071310 0x7fb344158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:29.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb3499b8640 1 -- 172.21.15.6:0/2215248577 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb34415df80 msgr2=0x7fb3440affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb3499b8640 1 --2- 172.21.15.6:0/2215248577 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb34415df80 0x7fb3440affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb3499b8640 1 -- 172.21.15.6:0/2215248577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb344071310 msgr2=0x7fb344158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb3499b8640 1 --2- 172.21.15.6:0/2215248577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb344071310 0x7fb344158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.263 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.260+0000 7fb3499b8640 1 -- 172.21.15.6:0/2215248577 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb340009400 con 0x7fb34406e3d0 2023-11-24T06:15:29.264 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.261+0000 7fb3499b8640 1 --2- 172.21.15.6:0/2215248577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb34406e3d0 0x7fb34415f8a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb340009890 tx=0x7fb340004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.261+0000 7fb33a7fc640 1 -- 172.21.15.6:0/2215248577 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34003e040 con 0x7fb34406e3d0 2023-11-24T06:15:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.261+0000 7fb33a7fc640 1 -- 172.21.15.6:0/2215248577 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb340039cd0 con 0x7fb34406e3d0 2023-11-24T06:15:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.261+0000 7fb33a7fc640 1 -- 172.21.15.6:0/2215248577 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340044630 con 0x7fb34406e3d0 2023-11-24T06:15:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.261+0000 7fb34a9ba640 1 -- 172.21.15.6:0/2215248577 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3440afaa0 con 0x7fb34406e3d0 2023-11-24T06:15:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.261+0000 7fb34a9ba640 1 -- 172.21.15.6:0/2215248577 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34415fde0 con 0x7fb34406e3d0 2023-11-24T06:15:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.262+0000 7fb33a7fc640 1 -- 172.21.15.6:0/2215248577 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb340044a90 con 0x7fb34406e3d0 2023-11-24T06:15:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.262+0000 7fb33a7fc640 1 -- 172.21.15.6:0/2215248577 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fb34004d840 con 0x7fb34406e3d0 2023-11-24T06:15:29.265 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.262+0000 7fb34a9ba640 1 -- 172.21.15.6:0/2215248577 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34415ffd0 con 0x7fb34406e3d0 2023-11-24T06:15:29.266 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.265+0000 7fb33a7fc640 1 -- 172.21.15.6:0/2215248577 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fb340049070 con 0x7fb34406e3d0 2023-11-24T06:15:29.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.344+0000 7fb34a9ba640 1 -- 172.21.15.6:0/2215248577 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "388d7df5-5c6c-45db-832b-a306b901fe33", "id": 3} v 0) v1 -- 0x7fb34404e840 con 0x7fb34406e3d0 2023-11-24T06:15:29.353 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.351+0000 7fb33a7fc640 1 -- 172.21.15.6:0/2215248577 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "388d7df5-5c6c-45db-832b-a306b901fe33", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb340034070 con 0x7fb34406e3d0 2023-11-24T06:15:29.353 INFO:teuthology.orchestra.run.smithi006.stdout:3 2023-11-24T06:15:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.353+0000 7fb34a9ba640 1 -- 172.21.15.6:0/2215248577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb34406e3d0 msgr2=0x7fb34415f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.353+0000 7fb34a9ba640 1 --2- 172.21.15.6:0/2215248577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb34406e3d0 0x7fb34415f8a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb340009890 tx=0x7fb340004730 comp rx=0 tx=0).stop 2023-11-24T06:15:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.353+0000 7fb34a9ba640 1 -- 172.21.15.6:0/2215248577 shutdown_connections 2023-11-24T06:15:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.353+0000 7fb34a9ba640 1 --2- 172.21.15.6:0/2215248577 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb34415df80 0x7fb3440affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.353+0000 7fb34a9ba640 1 --2- 172.21.15.6:0/2215248577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb344071310 0x7fb344158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.353+0000 7fb34a9ba640 1 --2- 172.21.15.6:0/2215248577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb34406e3d0 0x7fb34415f8a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.353+0000 7fb34a9ba640 1 -- 172.21.15.6:0/2215248577 >> 172.21.15.6:0/2215248577 conn(0x7fb3440b15c0 msgr2=0x7fb3440cfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.353+0000 7fb34a9ba640 1 -- 172.21.15.6:0/2215248577 shutdown_connections 2023-11-24T06:15:29.355 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.353+0000 7fb34a9ba640 1 -- 172.21.15.6:0/2215248577 wait complete. 2023-11-24T06:15:29.370 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new c12b3cb8-7a92-4dd8-9499-3ee17f6a2934 4 2023-11-24T06:15:29.443 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:29.441+0000 7f3a1d62e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T06:15:29.612 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:29.610+0000 7f3a1d62e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T06:15:29.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.613+0000 7f963c490640 1 Processor -- start 2023-11-24T06:15:29.615 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.613+0000 7f963c490640 1 -- start start 2023-11-24T06:15:29.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.615+0000 7f963c490640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9634143ff0 0x7f963406e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:29.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.615+0000 7f963c490640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96340c79a0 con 0x7f963406ec40 2023-11-24T06:15:29.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.615+0000 7f963c490640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96340c7ed0 con 0x7f9634144650 2023-11-24T06:15:29.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.615+0000 7f963c490640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963406f0d0 con 0x7f9634143ff0 2023-11-24T06:15:29.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.615+0000 7f9639a2b640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f9634144650 0x7f96340c5ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.6:40484/0 (socket says 172.21.15.6:40484) 2023-11-24T06:15:29.617 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.615+0000 7f9639a2b640 1 -- 172.21.15.6:0/2737048468 learned_addr learned my addr 172.21.15.6:0/2737048468 (peer_addr_for_me v1:172.21.15.6:0/0) 2023-11-24T06:15:29.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.615+0000 7f963a22c640 1 --2- 172.21.15.6:0/2737048468 >> v2:172.21.15.6:3300/0 conn(0x7f9634143ff0 0x7f963406e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:29.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.616+0000 7f963a22c640 1 -- 172.21.15.6:0/2737048468 >> v1:172.21.15.6:6789/0 conn(0x7f9634144650 legacy=0x7f96340c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.616+0000 7f963922a640 1 -- 172.21.15.6:0/2737048468 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943866088 0 0) 0x7f96340c7ed0 con 0x7f9634144650 2023-11-24T06:15:29.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.616+0000 7f963a22c640 1 -- 172.21.15.6:0/2737048468 >> v1:172.21.15.5:6789/0 conn(0x7f963406ec40 legacy=0x7f96340c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.618 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.616+0000 7f963a22c640 1 -- 172.21.15.6:0/2737048468 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96340ced50 con 0x7f9634143ff0 2023-11-24T06:15:29.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.617+0000 7f963a22c640 1 --2- 172.21.15.6:0/2737048468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9634143ff0 0x7f963406e700 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f96300097a0 tx=0x7f963002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24a673f3c575936 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:29.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.617+0000 7f963922a640 1 -- 172.21.15.6:0/2737048468 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630030610 con 0x7f9634143ff0 2023-11-24T06:15:29.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.617+0000 7f963922a640 1 -- 172.21.15.6:0/2737048468 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9630030770 con 0x7f9634143ff0 2023-11-24T06:15:29.619 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.617+0000 7f963922a640 1 -- 172.21.15.6:0/2737048468 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9630030b00 con 0x7f9634143ff0 2023-11-24T06:15:29.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.618+0000 7f963c490640 1 -- 172.21.15.6:0/2737048468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9634143ff0 msgr2=0x7f963406e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.618+0000 7f963c490640 1 --2- 172.21.15.6:0/2737048468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9634143ff0 0x7f963406e700 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f96300097a0 tx=0x7f963002b960 comp rx=0 tx=0).stop 2023-11-24T06:15:29.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.618+0000 7f963c490640 1 -- 172.21.15.6:0/2737048468 shutdown_connections 2023-11-24T06:15:29.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.618+0000 7f963c490640 1 --2- 172.21.15.6:0/2737048468 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9634143ff0 0x7f963406e700 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.620 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.618+0000 7f963c490640 1 -- 172.21.15.6:0/2737048468 >> 172.21.15.6:0/2737048468 conn(0x7f96340661f0 msgr2=0x7f96340665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:29.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.618+0000 7f963c490640 1 -- 172.21.15.6:0/2737048468 shutdown_connections 2023-11-24T06:15:29.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.618+0000 7f963c490640 1 -- 172.21.15.6:0/2737048468 wait complete. 2023-11-24T06:15:29.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.620+0000 7f963c490640 1 Processor -- start 2023-11-24T06:15:29.621 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.620+0000 7f963c490640 1 -- start start 2023-11-24T06:15:29.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f963c490640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f963406ec40 0x7f9634141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:29.623 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f963c490640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9634143ff0 0x7f96341e5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:29.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f963c490640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9634144650 0x7f96341e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:29.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f963c490640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96341f1ed0 con 0x7f9634144650 2023-11-24T06:15:29.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f963c490640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96341f15c0 con 0x7f963406ec40 2023-11-24T06:15:29.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f963c490640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96341e6430 con 0x7f9634143ff0 2023-11-24T06:15:29.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f963a22c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f963406ec40 0x7f9634141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:29.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f9639a2b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9634143ff0 0x7f96341e5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:29.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f963a22c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f963406ec40 0x7f9634141fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:40404/0 (socket says 172.21.15.6:40404) 2023-11-24T06:15:29.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f963a22c640 1 -- 172.21.15.6:0/2132301289 learned_addr learned my addr 172.21.15.6:0/2132301289 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-24T06:15:29.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f963aa2d640 1 --2- 172.21.15.6:0/2132301289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9634144650 0x7f96341e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:29.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f9639a2b640 1 -- 172.21.15.6:0/2132301289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f963406ec40 msgr2=0x7f9634141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.624 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f9639a2b640 1 --2- 172.21.15.6:0/2132301289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f963406ec40 0x7f9634141fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f9639a2b640 1 -- 172.21.15.6:0/2132301289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9634144650 msgr2=0x7f96341e5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f9639a2b640 1 --2- 172.21.15.6:0/2132301289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9634144650 0x7f96341e5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f9639a2b640 1 -- 172.21.15.6:0/2132301289 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9630009400 con 0x7f9634143ff0 2023-11-24T06:15:29.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.621+0000 7f963a22c640 1 --2- 172.21.15.6:0/2132301289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f963406ec40 0x7f9634141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:15:29.625 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.622+0000 7f9639a2b640 1 --2- 172.21.15.6:0/2132301289 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9634143ff0 0x7f96341e5900 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f962400c3e0 tx=0x7f962400c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:29.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.622+0000 7f962affd640 1 -- 172.21.15.6:0/2132301289 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96240074e0 con 0x7f9634143ff0 2023-11-24T06:15:29.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.622+0000 7f962affd640 1 -- 172.21.15.6:0/2132301289 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9624007640 con 0x7f9634143ff0 2023-11-24T06:15:29.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.622+0000 7f962affd640 1 -- 172.21.15.6:0/2132301289 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624004a20 con 0x7f9634143ff0 2023-11-24T06:15:29.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.622+0000 7f963c490640 1 -- 172.21.15.6:0/2132301289 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96341f1810 con 0x7f9634143ff0 2023-11-24T06:15:29.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.622+0000 7f963c490640 1 -- 172.21.15.6:0/2132301289 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96341e4c80 con 0x7f9634143ff0 2023-11-24T06:15:29.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.622+0000 7f962affd640 1 -- 172.21.15.6:0/2132301289 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f962401d3f0 con 0x7f9634143ff0 2023-11-24T06:15:29.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.623+0000 7f962affd640 1 -- 172.21.15.6:0/2132301289 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f962401dbb0 con 0x7f9634143ff0 2023-11-24T06:15:29.626 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.623+0000 7f963c490640 1 -- 172.21.15.6:0/2132301289 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96341f2010 con 0x7f9634143ff0 2023-11-24T06:15:29.627 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.626+0000 7f962affd640 1 -- 172.21.15.6:0/2132301289 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f96341f2010 con 0x7f9634143ff0 2023-11-24T06:15:29.669 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:29.667+0000 7f1d2dc3b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T06:15:29.706 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.704+0000 7f963c490640 1 -- 172.21.15.6:0/2132301289 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c12b3cb8-7a92-4dd8-9499-3ee17f6a2934", "id": 4} v 0) v1 -- 0x7f96340b7260 con 0x7f9634143ff0 2023-11-24T06:15:29.712 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.710+0000 7f962affd640 1 -- 172.21.15.6:0/2132301289 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c12b3cb8-7a92-4dd8-9499-3ee17f6a2934", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f962401d6a0 con 0x7f9634143ff0 2023-11-24T06:15:29.713 INFO:teuthology.orchestra.run.smithi006.stdout:4 2023-11-24T06:15:29.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.712+0000 7f963c490640 1 -- 172.21.15.6:0/2132301289 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9634143ff0 msgr2=0x7f96341e5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.712+0000 7f963c490640 1 --2- 172.21.15.6:0/2132301289 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9634143ff0 0x7f96341e5900 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f962400c3e0 tx=0x7f962400c8a0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.712+0000 7f963c490640 1 -- 172.21.15.6:0/2132301289 shutdown_connections 2023-11-24T06:15:29.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.712+0000 7f963c490640 1 --2- 172.21.15.6:0/2132301289 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9634143ff0 0x7f96341e5900 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.712+0000 7f963c490640 1 --2- 172.21.15.6:0/2132301289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9634144650 0x7f96341e5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.712+0000 7f963c490640 1 --2- 172.21.15.6:0/2132301289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f963406ec40 0x7f9634141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.712+0000 7f963c490640 1 -- 172.21.15.6:0/2132301289 >> 172.21.15.6:0/2132301289 conn(0x7f96340661f0 msgr2=0x7f96340665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:29.714 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.712+0000 7f963c490640 1 -- 172.21.15.6:0/2132301289 shutdown_connections 2023-11-24T06:15:29.715 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.712+0000 7f963c490640 1 -- 172.21.15.6:0/2132301289 wait complete. 2023-11-24T06:15:29.731 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0 5 2023-11-24T06:15:29.749 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:29.747+0000 7f3a1d62e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T06:15:29.805 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:29.803+0000 7f3a1d62e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T06:15:29.842 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:29.840+0000 7f1d2dc3b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T06:15:29.920 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:29.918+0000 7f3a1d62e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:15:29.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.923+0000 7fa3a7059640 1 Processor -- start 2023-11-24T06:15:29.925 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.923+0000 7fa3a7059640 1 -- start start 2023-11-24T06:15:29.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.925+0000 7fa3a7059640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa3a01618d0 0x7fa3a0070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:29.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.925+0000 7fa3a7059640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa3a0071310 0x7fa3a006a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:29.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.925+0000 7fa3a7059640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3a00b3290 con 0x7fa3a00b2eb0 2023-11-24T06:15:29.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.925+0000 7fa3a7059640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a0161d10 con 0x7fa3a0071310 2023-11-24T06:15:29.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.925+0000 7fa3a7059640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3a0071780 con 0x7fa3a01618d0 2023-11-24T06:15:29.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.925+0000 7fa3a5856640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa3a01618d0 0x7fa3a0070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:29.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.925+0000 7fa3a5856640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa3a01618d0 0x7fa3a0070dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46912/0 (socket says 172.21.15.6:46912) 2023-11-24T06:15:29.928 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.925+0000 7fa3a5856640 1 -- 172.21.15.6:0/3041314984 learned_addr learned my addr 172.21.15.6:0/3041314984 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-24T06:15:29.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.926+0000 7fa3a6858640 1 --2- 172.21.15.6:0/3041314984 >> v2:172.21.15.5:3300/0 conn(0x7fa3a0071310 0x7fa3a006a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:29.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.926+0000 7fa3a5856640 1 -- 172.21.15.6:0/3041314984 >> v2:172.21.15.5:3300/0 conn(0x7fa3a0071310 msgr2=0x7fa3a006a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:29.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.926+0000 7fa3a5856640 1 --2- 172.21.15.6:0/3041314984 >> v2:172.21.15.5:3300/0 conn(0x7fa3a0071310 0x7fa3a006a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.926+0000 7fa3a5856640 1 -- 172.21.15.6:0/3041314984 >> v1:172.21.15.5:6789/0 conn(0x7fa3a00b2eb0 legacy=0x7fa3a015d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.926+0000 7fa3a5856640 1 -- 172.21.15.6:0/3041314984 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a015e350 con 0x7fa3a01618d0 2023-11-24T06:15:29.929 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.927+0000 7fa3a5856640 1 --2- 172.21.15.6:0/3041314984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3a01618d0 0x7fa3a0070dd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00d05f0 tx=0x7fa39002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ef6855eb983a2e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:29.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.927+0000 7fa3a5055640 1 -- 172.21.15.6:0/3041314984 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390030860 con 0x7fa3a01618d0 2023-11-24T06:15:29.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.927+0000 7fa3a5055640 1 -- 172.21.15.6:0/3041314984 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3900309c0 con 0x7fa3a01618d0 2023-11-24T06:15:29.930 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.927+0000 7fa3a5055640 1 -- 172.21.15.6:0/3041314984 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390030d50 con 0x7fa3a01618d0 2023-11-24T06:15:29.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.928+0000 7fa3a7059640 1 -- 172.21.15.6:0/3041314984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3a01618d0 msgr2=0x7fa3a0070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.928+0000 7fa3a7059640 1 --2- 172.21.15.6:0/3041314984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3a01618d0 0x7fa3a0070dd0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00d05f0 tx=0x7fa39002b9a0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.928+0000 7fa3a7059640 1 -- 172.21.15.6:0/3041314984 shutdown_connections 2023-11-24T06:15:29.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.928+0000 7fa3a7059640 1 --2- 172.21.15.6:0/3041314984 >> v2:172.21.15.5:3300/0 conn(0x7fa3a0071310 0x7fa3a006a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.928+0000 7fa3a7059640 1 --2- 172.21.15.6:0/3041314984 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3a01618d0 0x7fa3a0070dd0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.928+0000 7fa3a7059640 1 -- 172.21.15.6:0/3041314984 >> 172.21.15.6:0/3041314984 conn(0x7fa3a00b15c0 msgr2=0x7fa3a00b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:29.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.928+0000 7fa3a7059640 1 -- 172.21.15.6:0/3041314984 shutdown_connections 2023-11-24T06:15:29.931 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.928+0000 7fa3a7059640 1 -- 172.21.15.6:0/3041314984 wait complete. 2023-11-24T06:15:29.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.931+0000 7fa3a7059640 1 Processor -- start 2023-11-24T06:15:29.932 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.931+0000 7fa3a7059640 1 -- start start 2023-11-24T06:15:29.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.932+0000 7fa3a7059640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3a0071310 0x7fa3a00ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:29.934 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.932+0000 7fa3a7059640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3a00b2eb0 0x7fa3a0141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:29.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.932+0000 7fa3a7059640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a01618d0 0x7fa3a0144400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:29.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.932+0000 7fa3a7059640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a0156f70 con 0x7fa3a01618d0 2023-11-24T06:15:29.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.932+0000 7fa3a7059640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a0142370 con 0x7fa3a00b2eb0 2023-11-24T06:15:29.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.932+0000 7fa3a7059640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a00cc4f0 con 0x7fa3a0071310 2023-11-24T06:15:29.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.932+0000 7fa3a6057640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3a0071310 0x7fa3a00ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:29.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.933+0000 7fa3a5856640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3a00b2eb0 0x7fa3a0141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:29.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.933+0000 7fa3a6057640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3a0071310 0x7fa3a00ceee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:34036/0 (socket says 172.21.15.6:34036) 2023-11-24T06:15:29.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.933+0000 7fa3a5856640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3a00b2eb0 0x7fa3a0141b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46924/0 (socket says 172.21.15.6:46924) 2023-11-24T06:15:29.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.933+0000 7fa3a6057640 1 -- 172.21.15.6:0/1133042284 learned_addr learned my addr 172.21.15.6:0/1133042284 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-24T06:15:29.935 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.933+0000 7fa3a6858640 1 --2- 172.21.15.6:0/1133042284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a01618d0 0x7fa3a0144400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:29.936 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.933+0000 7fa3a5856640 1 -- 172.21.15.6:0/1133042284 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3a0071310 msgr2=0x7fa3a00ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.933+0000 7fa3a5856640 1 --2- 172.21.15.6:0/1133042284 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3a0071310 0x7fa3a00ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.933+0000 7fa3a5856640 1 -- 172.21.15.6:0/1133042284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a01618d0 msgr2=0x7fa3a0144400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:29.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.933+0000 7fa3a5856640 1 --2- 172.21.15.6:0/1133042284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a01618d0 0x7fa3a0144400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:29.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.933+0000 7fa3a5856640 1 -- 172.21.15.6:0/1133042284 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3900094d0 con 0x7fa3a00b2eb0 2023-11-24T06:15:29.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.933+0000 7fa3a5856640 1 --2- 172.21.15.6:0/1133042284 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3a00b2eb0 0x7fa3a0141b50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa390033000 tx=0x7fa390004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:29.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.934+0000 7fa396ffd640 1 -- 172.21.15.6:0/1133042284 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390039c90 con 0x7fa3a00b2eb0 2023-11-24T06:15:29.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.934+0000 7fa396ffd640 1 -- 172.21.15.6:0/1133042284 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa390039e10 con 0x7fa3a00b2eb0 2023-11-24T06:15:29.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.934+0000 7fa396ffd640 1 -- 172.21.15.6:0/1133042284 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa390043640 con 0x7fa3a00b2eb0 2023-11-24T06:15:29.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.934+0000 7fa3a7059640 1 -- 172.21.15.6:0/1133042284 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a0142090 con 0x7fa3a00b2eb0 2023-11-24T06:15:29.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.934+0000 7fa3a7059640 1 -- 172.21.15.6:0/1133042284 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a01425c0 con 0x7fa3a00b2eb0 2023-11-24T06:15:29.937 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.934+0000 7fa396ffd640 1 -- 172.21.15.6:0/1133042284 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa390043aa0 con 0x7fa3a00b2eb0 2023-11-24T06:15:29.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.934+0000 7fa396ffd640 1 -- 172.21.15.6:0/1133042284 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fa39003e070 con 0x7fa3a00b2eb0 2023-11-24T06:15:29.938 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.935+0000 7fa3a7059640 1 -- 172.21.15.6:0/1133042284 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a004e840 con 0x7fa3a00b2eb0 2023-11-24T06:15:29.940 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:29.938+0000 7fa396ffd640 1 -- 172.21.15.6:0/1133042284 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fa390034070 con 0x7fa3a00b2eb0 2023-11-24T06:15:29.989 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:29.987+0000 7f1d2dc3b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T06:15:30.020 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.018+0000 7fa3a7059640 1 -- 172.21.15.6:0/1133042284 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0", "id": 5} v 0) v1 -- 0x7fa3a006b9f0 con 0x7fa3a00b2eb0 2023-11-24T06:15:30.027 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.025+0000 7fa396ffd640 1 -- 172.21.15.6:0/1133042284 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa39004d460 con 0x7fa3a00b2eb0 2023-11-24T06:15:30.027 INFO:teuthology.orchestra.run.smithi006.stdout:5 2023-11-24T06:15:30.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.026+0000 7fa3a7059640 1 -- 172.21.15.6:0/1133042284 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3a00b2eb0 msgr2=0x7fa3a0141b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:30.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.026+0000 7fa3a7059640 1 --2- 172.21.15.6:0/1133042284 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3a00b2eb0 0x7fa3a0141b50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa390033000 tx=0x7fa390004730 comp rx=0 tx=0).stop 2023-11-24T06:15:30.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.027+0000 7fa3a7059640 1 -- 172.21.15.6:0/1133042284 shutdown_connections 2023-11-24T06:15:30.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.027+0000 7fa3a7059640 1 --2- 172.21.15.6:0/1133042284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3a01618d0 0x7fa3a0144400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.027+0000 7fa3a7059640 1 --2- 172.21.15.6:0/1133042284 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3a00b2eb0 0x7fa3a0141b50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.027+0000 7fa3a7059640 1 --2- 172.21.15.6:0/1133042284 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3a0071310 0x7fa3a00ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.027+0000 7fa3a7059640 1 -- 172.21.15.6:0/1133042284 >> 172.21.15.6:0/1133042284 conn(0x7fa3a00b15c0 msgr2=0x7fa3a00ad6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:30.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.027+0000 7fa3a7059640 1 -- 172.21.15.6:0/1133042284 shutdown_connections 2023-11-24T06:15:30.029 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.027+0000 7fa3a7059640 1 -- 172.21.15.6:0/1133042284 wait complete. 2023-11-24T06:15:30.038 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:30.036+0000 7f3a1d62e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T06:15:30.045 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new f31a6b12-4ba8-494e-b00d-79c187dc5aad 6 2023-11-24T06:15:30.050 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:30.046+0000 7f1d2dc3b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T06:15:30.166 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:30.164+0000 7f1d2dc3b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:15:30.206 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:30.204+0000 7f3a1d62e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:15:30.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.241+0000 7f4e871dc640 1 Processor -- start 2023-11-24T06:15:30.243 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.241+0000 7f4e871dc640 1 -- start start 2023-11-24T06:15:30.245 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.243+0000 7f4e871dc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e80071300 0x7f4e801619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.243+0000 7f4e871dc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4e8006cf30 0x7f4e8006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.243+0000 7f4e871dc640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e80161ef0 con 0x7f4e8015d440 2023-11-24T06:15:30.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.243+0000 7f4e871dc640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e800cf500 con 0x7f4e8006cf30 2023-11-24T06:15:30.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.243+0000 7f4e871dc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e800ce8f0 con 0x7f4e80071300 2023-11-24T06:15:30.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.243+0000 7f4e7ffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e80071300 0x7f4e801619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.243+0000 7f4e7ffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e80071300 0x7f4e801619b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:34046/0 (socket says 172.21.15.6:34046) 2023-11-24T06:15:30.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.243+0000 7f4e7ffff640 1 -- 172.21.15.6:0/4200333259 learned_addr learned my addr 172.21.15.6:0/4200333259 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-24T06:15:30.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.244+0000 7f4e85779640 1 --2- 172.21.15.6:0/4200333259 >> v2:172.21.15.5:3300/0 conn(0x7f4e8006cf30 0x7f4e8006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.244+0000 7f4e7ffff640 1 -- 172.21.15.6:0/4200333259 >> v2:172.21.15.5:3300/0 conn(0x7f4e8006cf30 msgr2=0x7f4e8006d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:30.246 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.244+0000 7f4e7ffff640 1 --2- 172.21.15.6:0/4200333259 >> v2:172.21.15.5:3300/0 conn(0x7f4e8006cf30 0x7f4e8006d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.244+0000 7f4e7ffff640 1 -- 172.21.15.6:0/4200333259 >> v1:172.21.15.5:6789/0 conn(0x7f4e8015d440 legacy=0x7f4e80070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:30.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.244+0000 7f4e7ffff640 1 -- 172.21.15.6:0/4200333259 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e800cebd0 con 0x7f4e80071300 2023-11-24T06:15:30.247 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.244+0000 7f4e85779640 1 --2- 172.21.15.6:0/4200333259 >> v2:172.21.15.5:3300/0 conn(0x7f4e8006cf30 0x7f4e8006d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:15:30.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.245+0000 7f4e7ffff640 1 --2- 172.21.15.6:0/4200333259 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e80071300 0x7f4e801619b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009870 tx=0x7f4e6c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f7b836aadf4d469 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:30.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.245+0000 7f4e7f7fe640 1 -- 172.21.15.6:0/4200333259 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c031640 con 0x7f4e80071300 2023-11-24T06:15:30.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.245+0000 7f4e7f7fe640 1 -- 172.21.15.6:0/4200333259 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c0317a0 con 0x7f4e80071300 2023-11-24T06:15:30.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.245+0000 7f4e7f7fe640 1 -- 172.21.15.6:0/4200333259 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c031b30 con 0x7f4e80071300 2023-11-24T06:15:30.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.246+0000 7f4e871dc640 1 -- 172.21.15.6:0/4200333259 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e80071300 msgr2=0x7f4e801619b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:30.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.246+0000 7f4e871dc640 1 --2- 172.21.15.6:0/4200333259 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e80071300 0x7f4e801619b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009870 tx=0x7f4e6c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.246+0000 7f4e871dc640 1 -- 172.21.15.6:0/4200333259 shutdown_connections 2023-11-24T06:15:30.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.246+0000 7f4e871dc640 1 --2- 172.21.15.6:0/4200333259 >> v2:172.21.15.5:3300/0 conn(0x7f4e8006cf30 0x7f4e8006d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.248 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.246+0000 7f4e871dc640 1 --2- 172.21.15.6:0/4200333259 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e80071300 0x7f4e801619b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.246+0000 7f4e871dc640 1 -- 172.21.15.6:0/4200333259 >> 172.21.15.6:0/4200333259 conn(0x7f4e800cf740 msgr2=0x7f4e800cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.246+0000 7f4e871dc640 1 -- 172.21.15.6:0/4200333259 shutdown_connections 2023-11-24T06:15:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.246+0000 7f4e871dc640 1 -- 172.21.15.6:0/4200333259 wait complete. 2023-11-24T06:15:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.247+0000 7f4e871dc640 1 Processor -- start 2023-11-24T06:15:30.249 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.248+0000 7f4e871dc640 1 -- start start 2023-11-24T06:15:30.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.248+0000 7f4e871dc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e8006cf30 0x7f4e801e89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e871dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e80071300 0x7f4e80156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.251 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e871dc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e8015d440 0x7f4e801e6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e871dc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e801e8550 con 0x7f4e80071300 2023-11-24T06:15:30.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e871dc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e80159f90 con 0x7f4e8006cf30 2023-11-24T06:15:30.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e871dc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e801e7940 con 0x7f4e8015d440 2023-11-24T06:15:30.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e85779640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e8015d440 0x7f4e801e6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e84f78640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e8006cf30 0x7f4e801e89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e85779640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e8015d440 0x7f4e801e6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:34056/0 (socket says 172.21.15.6:34056) 2023-11-24T06:15:30.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e85779640 1 -- 172.21.15.6:0/2585509539 learned_addr learned my addr 172.21.15.6:0/2585509539 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-24T06:15:30.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e7ffff640 1 --2- 172.21.15.6:0/2585509539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e80071300 0x7f4e80156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.252 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e85779640 1 -- 172.21.15.6:0/2585509539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e8006cf30 msgr2=0x7f4e801e89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:30.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e85779640 1 --2- 172.21.15.6:0/2585509539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e8006cf30 0x7f4e801e89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e85779640 1 -- 172.21.15.6:0/2585509539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e80071300 msgr2=0x7f4e80156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:30.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e85779640 1 --2- 172.21.15.6:0/2585509539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e80071300 0x7f4e80156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.249+0000 7f4e85779640 1 -- 172.21.15.6:0/2585509539 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c0094d0 con 0x7f4e8015d440 2023-11-24T06:15:30.253 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.250+0000 7f4e85779640 1 --2- 172.21.15.6:0/2585509539 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e8015d440 0x7f4e801e6da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800cef0 tx=0x7f4e7800a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:30.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.250+0000 7f4e7d7fa640 1 -- 172.21.15.6:0/2585509539 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7800d840 con 0x7f4e8015d440 2023-11-24T06:15:30.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.250+0000 7f4e7d7fa640 1 -- 172.21.15.6:0/2585509539 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7800d9c0 con 0x7f4e8015d440 2023-11-24T06:15:30.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.250+0000 7f4e7d7fa640 1 -- 172.21.15.6:0/2585509539 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7800dd70 con 0x7f4e8015d440 2023-11-24T06:15:30.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.250+0000 7f4e871dc640 1 -- 172.21.15.6:0/2585509539 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e801e7c20 con 0x7f4e8015d440 2023-11-24T06:15:30.254 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.250+0000 7f4e871dc640 1 -- 172.21.15.6:0/2585509539 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e801e81e0 con 0x7f4e8015d440 2023-11-24T06:15:30.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.250+0000 7f4e7d7fa640 1 -- 172.21.15.6:0/2585509539 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4e7801f740 con 0x7f4e8015d440 2023-11-24T06:15:30.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.250+0000 7f4e7d7fa640 1 -- 172.21.15.6:0/2585509539 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f4e78015da0 con 0x7f4e8015d440 2023-11-24T06:15:30.255 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.251+0000 7f4e871dc640 1 -- 172.21.15.6:0/2585509539 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e800ce130 con 0x7f4e8015d440 2023-11-24T06:15:30.257 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.255+0000 7f4e7d7fa640 1 -- 172.21.15.6:0/2585509539 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f4e7801a070 con 0x7f4e8015d440 2023-11-24T06:15:30.258 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:30.256+0000 7f3a1d62e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T06:15:30.287 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:30.285+0000 7f1d2dc3b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T06:15:30.335 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.333+0000 7f4e871dc640 1 -- 172.21.15.6:0/2585509539 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f31a6b12-4ba8-494e-b00d-79c187dc5aad", "id": 6} v 0) v1 -- 0x7f4e801e86c0 con 0x7f4e8015d440 2023-11-24T06:15:30.346 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.344+0000 7f4e7d7fa640 1 -- 172.21.15.6:0/2585509539 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f31a6b12-4ba8-494e-b00d-79c187dc5aad", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4e780157b0 con 0x7f4e8015d440 2023-11-24T06:15:30.346 INFO:teuthology.orchestra.run.smithi006.stdout:6 2023-11-24T06:15:30.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.345+0000 7f4e871dc640 1 -- 172.21.15.6:0/2585509539 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e8015d440 msgr2=0x7f4e801e6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:30.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.345+0000 7f4e871dc640 1 --2- 172.21.15.6:0/2585509539 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e8015d440 0x7f4e801e6da0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800cef0 tx=0x7f4e7800a710 comp rx=0 tx=0).stop 2023-11-24T06:15:30.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.345+0000 7f4e871dc640 1 -- 172.21.15.6:0/2585509539 shutdown_connections 2023-11-24T06:15:30.347 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.345+0000 7f4e871dc640 1 --2- 172.21.15.6:0/2585509539 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e8015d440 0x7f4e801e6da0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.345+0000 7f4e871dc640 1 --2- 172.21.15.6:0/2585509539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e80071300 0x7f4e80156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.345+0000 7f4e871dc640 1 --2- 172.21.15.6:0/2585509539 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e8006cf30 0x7f4e801e89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.345+0000 7f4e871dc640 1 -- 172.21.15.6:0/2585509539 >> 172.21.15.6:0/2585509539 conn(0x7f4e800cf740 msgr2=0x7f4e80070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:30.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.345+0000 7f4e871dc640 1 -- 172.21.15.6:0/2585509539 shutdown_connections 2023-11-24T06:15:30.348 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.345+0000 7f4e871dc640 1 -- 172.21.15.6:0/2585509539 wait complete. 2023-11-24T06:15:30.363 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --cluster ceph osd new f3ee2ac0-f1db-4dee-80cc-ce8998d7259e 7 2023-11-24T06:15:30.424 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:30.422+0000 7f3a1d62e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T06:15:30.459 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:30.457+0000 7f1d2dc3b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T06:15:30.512 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:30.510+0000 7f1d2dc3b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T06:15:30.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.552+0000 7f2556b4c640 1 Processor -- start 2023-11-24T06:15:30.554 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.552+0000 7f2556b4c640 1 -- start start 2023-11-24T06:15:30.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.554+0000 7f2556b4c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2550160bd0 0x7f25500b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.554+0000 7f2556b4c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f25500b48d0 0x7f25500b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.554+0000 7f2556b4c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25500b23e0 con 0x7f25500b3040 2023-11-24T06:15:30.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.554+0000 7f2556b4c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25500cdc70 con 0x7f25500b48d0 2023-11-24T06:15:30.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.554+0000 7f2556b4c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25500b2870 con 0x7f2550160bd0 2023-11-24T06:15:30.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.554+0000 7f25548e8640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2550160bd0 0x7f25500b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.555+0000 7f25548e8640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2550160bd0 0x7f25500b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.6:46948/0 (socket says 172.21.15.6:46948) 2023-11-24T06:15:30.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.555+0000 7f25548e8640 1 -- 172.21.15.6:0/3900457271 learned_addr learned my addr 172.21.15.6:0/3900457271 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-24T06:15:30.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.555+0000 7f254ffff640 1 --2- 172.21.15.6:0/3900457271 >> v2:172.21.15.5:3300/0 conn(0x7f25500b48d0 0x7f25500b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.557 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.555+0000 7f25548e8640 1 -- 172.21.15.6:0/3900457271 >> v2:172.21.15.5:3300/0 conn(0x7f25500b48d0 msgr2=0x7f25500b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:30.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.555+0000 7f25548e8640 1 --2- 172.21.15.6:0/3900457271 >> v2:172.21.15.5:3300/0 conn(0x7f25500b48d0 0x7f25500b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.555+0000 7f25548e8640 1 -- 172.21.15.6:0/3900457271 >> v1:172.21.15.6:6789/0 conn(0x7f25500b3040 legacy=0x7f2550166280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:30.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.555+0000 7f25548e8640 1 -- 172.21.15.6:0/3900457271 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25500d41b0 con 0x7f2550160bd0 2023-11-24T06:15:30.558 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.555+0000 7f254ffff640 1 --2- 172.21.15.6:0/3900457271 >> v2:172.21.15.5:3300/0 conn(0x7f25500b48d0 0x7f25500b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:15:30.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.556+0000 7f25548e8640 1 --2- 172.21.15.6:0/3900457271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2550160bd0 0x7f25500b1ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f25500cda60 tx=0x7f254402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7654d0161b0129fb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:30.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.556+0000 7f254f7fe640 1 -- 172.21.15.6:0/3900457271 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2544031870 con 0x7f2550160bd0 2023-11-24T06:15:30.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.556+0000 7f254f7fe640 1 -- 172.21.15.6:0/3900457271 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25440319d0 con 0x7f2550160bd0 2023-11-24T06:15:30.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.556+0000 7f254f7fe640 1 -- 172.21.15.6:0/3900457271 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2544031d60 con 0x7f2550160bd0 2023-11-24T06:15:30.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.557+0000 7f2556b4c640 1 -- 172.21.15.6:0/3900457271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2550160bd0 msgr2=0x7f25500b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:30.559 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.557+0000 7f2556b4c640 1 --2- 172.21.15.6:0/3900457271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2550160bd0 0x7f25500b1ea0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f25500cda60 tx=0x7f254402b910 comp rx=0 tx=0).stop 2023-11-24T06:15:30.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.557+0000 7f2556b4c640 1 -- 172.21.15.6:0/3900457271 shutdown_connections 2023-11-24T06:15:30.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.557+0000 7f2556b4c640 1 --2- 172.21.15.6:0/3900457271 >> v2:172.21.15.5:3300/0 conn(0x7f25500b48d0 0x7f25500b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.557+0000 7f2556b4c640 1 --2- 172.21.15.6:0/3900457271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2550160bd0 0x7f25500b1ea0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.557+0000 7f2556b4c640 1 -- 172.21.15.6:0/3900457271 >> 172.21.15.6:0/3900457271 conn(0x7f255009c5a0 msgr2=0x7f25500d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:30.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.557+0000 7f2556b4c640 1 -- 172.21.15.6:0/3900457271 shutdown_connections 2023-11-24T06:15:30.560 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.557+0000 7f2556b4c640 1 -- 172.21.15.6:0/3900457271 wait complete. 2023-11-24T06:15:30.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.559+0000 7f2556b4c640 1 Processor -- start 2023-11-24T06:15:30.561 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.559+0000 7f2556b4c640 1 -- start start 2023-11-24T06:15:30.562 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.560+0000 7f2556b4c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25500b3040 0x7f2550158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.560+0000 7f2556b4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25500b48d0 0x7f25500cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.560+0000 7f2556b4c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2550160bd0 0x7f25500cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.560+0000 7f2556b4c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25500cb760 con 0x7f25500b48d0 2023-11-24T06:15:30.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.560+0000 7f2556b4c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25500cbb80 con 0x7f2550160bd0 2023-11-24T06:15:30.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.560+0000 7f2556b4c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25500cadc0 con 0x7f25500b3040 2023-11-24T06:15:30.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.560+0000 7f25548e8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25500b3040 0x7f2550158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.560+0000 7f25548e8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25500b3040 0x7f2550158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.6:34058/0 (socket says 172.21.15.6:34058) 2023-11-24T06:15:30.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.560+0000 7f25548e8640 1 -- 172.21.15.6:0/2970915900 learned_addr learned my addr 172.21.15.6:0/2970915900 (peer_addr_for_me v2:172.21.15.6:0/0) 2023-11-24T06:15:30.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.561+0000 7f25550e9640 1 --2- 172.21.15.6:0/2970915900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2550160bd0 0x7f25500cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.563 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.561+0000 7f254ffff640 1 --2- 172.21.15.6:0/2970915900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25500b48d0 0x7f25500cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.561+0000 7f25548e8640 1 -- 172.21.15.6:0/2970915900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2550160bd0 msgr2=0x7f25500cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:30.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.561+0000 7f25548e8640 1 --2- 172.21.15.6:0/2970915900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2550160bd0 0x7f25500cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.564 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.561+0000 7f25548e8640 1 -- 172.21.15.6:0/2970915900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25500b48d0 msgr2=0x7f25500cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:30.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.561+0000 7f25548e8640 1 --2- 172.21.15.6:0/2970915900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25500b48d0 0x7f25500cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.561+0000 7f25548e8640 1 -- 172.21.15.6:0/2970915900 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25440094d0 con 0x7f25500b3040 2023-11-24T06:15:30.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.561+0000 7f25548e8640 1 --2- 172.21.15.6:0/2970915900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25500b3040 0x7f2550158a50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f25440090b0 tx=0x7f25440046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:30.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.562+0000 7f254d7fa640 1 -- 172.21.15.6:0/2970915900 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254403adf0 con 0x7f25500b3040 2023-11-24T06:15:30.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.562+0000 7f254d7fa640 1 -- 172.21.15.6:0/2970915900 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25440443f0 con 0x7f25500b3040 2023-11-24T06:15:30.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.562+0000 7f254d7fa640 1 -- 172.21.15.6:0/2970915900 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25440447a0 con 0x7f25500b3040 2023-11-24T06:15:30.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.562+0000 7f2556b4c640 1 -- 172.21.15.6:0/2970915900 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25500cb0a0 con 0x7f25500b3040 2023-11-24T06:15:30.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.562+0000 7f2556b4c640 1 -- 172.21.15.6:0/2970915900 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25500c9b60 con 0x7f25500b3040 2023-11-24T06:15:30.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.562+0000 7f254d7fa640 1 -- 172.21.15.6:0/2970915900 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2544044c00 con 0x7f25500b3040 2023-11-24T06:15:30.565 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.562+0000 7f254d7fa640 1 -- 172.21.15.6:0/2970915900 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f254402d040 con 0x7f25500b3040 2023-11-24T06:15:30.566 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.563+0000 7f2556b4c640 1 -- 172.21.15.6:0/2970915900 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25500c7e70 con 0x7f25500b3040 2023-11-24T06:15:30.569 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.567+0000 7f254d7fa640 1 -- 172.21.15.6:0/2970915900 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f25500c7e70 con 0x7f25500b3040 2023-11-24T06:15:30.647 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.645+0000 7f2556b4c640 1 -- 172.21.15.6:0/2970915900 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f3ee2ac0-f1db-4dee-80cc-ce8998d7259e", "id": 7} v 0) v1 -- 0x7f25500a1210 con 0x7f25500b3040 2023-11-24T06:15:30.657 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.655+0000 7f254d7fa640 1 -- 172.21.15.6:0/2970915900 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f3ee2ac0-f1db-4dee-80cc-ce8998d7259e", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2544052070 con 0x7f25500b3040 2023-11-24T06:15:30.657 INFO:teuthology.orchestra.run.smithi006.stdout:7 2023-11-24T06:15:30.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.657+0000 7f2556b4c640 1 -- 172.21.15.6:0/2970915900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25500b3040 msgr2=0x7f2550158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:30.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.657+0000 7f2556b4c640 1 --2- 172.21.15.6:0/2970915900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25500b3040 0x7f2550158a50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f25440090b0 tx=0x7f25440046c0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.657+0000 7f2556b4c640 1 -- 172.21.15.6:0/2970915900 shutdown_connections 2023-11-24T06:15:30.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.657+0000 7f2556b4c640 1 --2- 172.21.15.6:0/2970915900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2550160bd0 0x7f25500cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.657+0000 7f2556b4c640 1 --2- 172.21.15.6:0/2970915900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25500b48d0 0x7f25500cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.657+0000 7f2556b4c640 1 --2- 172.21.15.6:0/2970915900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f25500b3040 0x7f2550158a50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.657+0000 7f2556b4c640 1 -- 172.21.15.6:0/2970915900 >> 172.21.15.6:0/2970915900 conn(0x7f255009c5a0 msgr2=0x7f25500cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:30.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.657+0000 7f2556b4c640 1 -- 172.21.15.6:0/2970915900 shutdown_connections 2023-11-24T06:15:30.659 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:15:30.657+0000 7f2556b4c640 1 -- 172.21.15.6:0/2970915900 wait complete. 2023-11-24T06:15:30.676 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-24T06:15:30.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-24T06:15:30.684 INFO:tasks.ceph.osd.0:Started 2023-11-24T06:15:30.684 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-24T06:15:30.685 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-24T06:15:30.685 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:30.682+0000 7f1d2dc3b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T06:15:30.693 INFO:tasks.ceph.osd.1:Started 2023-11-24T06:15:30.693 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-24T06:15:30.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-24T06:15:30.700 INFO:tasks.ceph.osd.2:Started 2023-11-24T06:15:30.700 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-24T06:15:30.700 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-24T06:15:30.706 INFO:tasks.ceph.osd.3:Started 2023-11-24T06:15:30.706 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-24T06:15:30.706 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-24T06:15:30.711 INFO:tasks.ceph.osd.4:Started 2023-11-24T06:15:30.712 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-24T06:15:30.712 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-24T06:15:30.719 INFO:tasks.ceph.osd.5:Started 2023-11-24T06:15:30.720 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-24T06:15:30.720 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-24T06:15:30.725 INFO:tasks.ceph.osd.6:Started 2023-11-24T06:15:30.725 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-24T06:15:30.725 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-24T06:15:30.731 INFO:tasks.ceph.osd.7:Started 2023-11-24T06:15:30.732 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:15:30.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.965+0000 7f234c997640 1 Processor -- start 2023-11-24T06:15:30.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.965+0000 7f234c997640 1 -- start start 2023-11-24T06:15:30.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.966+0000 7f234c997640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f234815d4d0 0x7f23480cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.966+0000 7f234c997640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23480ce260 0x7f23480d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.966+0000 7f234c997640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234815d8b0 con 0x7f23480ce6c0 2023-11-24T06:15:30.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.966+0000 7f234c997640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23480b91b0 con 0x7f234815d4d0 2023-11-24T06:15:30.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.966+0000 7f234c997640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23480cdfb0 con 0x7f23480ce260 2023-11-24T06:15:30.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.967+0000 7f234659c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f234815d4d0 0x7f23480cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.967+0000 7f234659c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f234815d4d0 0x7f23480cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46278/0 (socket says 172.21.15.5:46278) 2023-11-24T06:15:30.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.967+0000 7f234659c640 1 -- 172.21.15.5:0/1957533449 learned_addr learned my addr 172.21.15.5:0/1957533449 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:30.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.967+0000 7f2345d9b640 1 --2- 172.21.15.5:0/1957533449 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23480ce260 0x7f23480d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.967+0000 7f234659c640 1 -- 172.21.15.5:0/1957533449 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23480ce260 msgr2=0x7f23480d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:30.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.967+0000 7f234659c640 1 --2- 172.21.15.5:0/1957533449 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23480ce260 0x7f23480d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.967+0000 7f234659c640 1 -- 172.21.15.5:0/1957533449 >> v1:172.21.15.6:6789/0 conn(0x7f23480ce6c0 legacy=0x7f23480b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:30.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.967+0000 7f234659c640 1 -- 172.21.15.5:0/1957533449 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23480b2400 con 0x7f234815d4d0 2023-11-24T06:15:30.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.968+0000 7f234659c640 1 --2- 172.21.15.5:0/1957533449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234815d4d0 0x7f23480cda70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2338009870 tx=0x7f233802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f3ce46d707ed451 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:30.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.968+0000 7f234559a640 1 -- 172.21.15.5:0/1957533449 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338030840 con 0x7f234815d4d0 2023-11-24T06:15:30.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.968+0000 7f234559a640 1 -- 172.21.15.5:0/1957533449 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23380309a0 con 0x7f234815d4d0 2023-11-24T06:15:30.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.969+0000 7f234559a640 1 -- 172.21.15.5:0/1957533449 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338030d30 con 0x7f234815d4d0 2023-11-24T06:15:30.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.969+0000 7f234c997640 1 -- 172.21.15.5:0/1957533449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234815d4d0 msgr2=0x7f23480cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:30.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.969+0000 7f234c997640 1 --2- 172.21.15.5:0/1957533449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234815d4d0 0x7f23480cda70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2338009870 tx=0x7f233802b980 comp rx=0 tx=0).stop 2023-11-24T06:15:30.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.969+0000 7f234c997640 1 -- 172.21.15.5:0/1957533449 shutdown_connections 2023-11-24T06:15:30.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.969+0000 7f234c997640 1 --2- 172.21.15.5:0/1957533449 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23480ce260 0x7f23480d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.969+0000 7f234c997640 1 --2- 172.21.15.5:0/1957533449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f234815d4d0 0x7f23480cda70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.969+0000 7f234c997640 1 -- 172.21.15.5:0/1957533449 >> 172.21.15.5:0/1957533449 conn(0x7f2348071f90 msgr2=0x7f23480723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:30.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.969+0000 7f234c997640 1 -- 172.21.15.5:0/1957533449 shutdown_connections 2023-11-24T06:15:30.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.969+0000 7f234c997640 1 -- 172.21.15.5:0/1957533449 wait complete. 2023-11-24T06:15:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.971+0000 7f234c997640 1 Processor -- start 2023-11-24T06:15:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.971+0000 7f234c997640 1 -- start start 2023-11-24T06:15:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.972+0000 7f234c997640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23480ce260 0x7f23480c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.972+0000 7f234c997640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23480ce6c0 0x7f23480c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.972+0000 7f234c997640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f234815d4d0 0x7f23480c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:30.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.972+0000 7f234c997640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23481f2eb0 con 0x7f23480ce260 2023-11-24T06:15:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.972+0000 7f234c997640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23480cc6f0 con 0x7f234815d4d0 2023-11-24T06:15:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.972+0000 7f234c997640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23480cbae0 con 0x7f23480ce6c0 2023-11-24T06:15:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.972+0000 7f234659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23480ce260 0x7f23480c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.972+0000 7f234659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23480ce260 0x7f23480c97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46284/0 (socket says 172.21.15.5:46284) 2023-11-24T06:15:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.972+0000 7f234659c640 1 -- 172.21.15.5:0/4140324851 learned_addr learned my addr 172.21.15.5:0/4140324851 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.972+0000 7f2345d9b640 1 --2- 172.21.15.5:0/4140324851 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23480ce6c0 0x7f23480c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.972+0000 7f2346d9d640 1 --2- 172.21.15.5:0/4140324851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f234815d4d0 0x7f23480c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.973+0000 7f234659c640 1 -- 172.21.15.5:0/4140324851 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23480ce6c0 msgr2=0x7f23480c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.973+0000 7f234659c640 1 --2- 172.21.15.5:0/4140324851 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23480ce6c0 0x7f23480c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.973+0000 7f234659c640 1 -- 172.21.15.5:0/4140324851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f234815d4d0 msgr2=0x7f23480c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.973+0000 7f234659c640 1 --2- 172.21.15.5:0/4140324851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f234815d4d0 0x7f23480c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:30.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.973+0000 7f234659c640 1 -- 172.21.15.5:0/4140324851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23380094d0 con 0x7f23480ce260 2023-11-24T06:15:30.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.973+0000 7f234659c640 1 --2- 172.21.15.5:0/4140324851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23480ce260 0x7f23480c97b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2338000c00 tx=0x7f2338004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:30.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.973+0000 7f2336ffd640 1 -- 172.21.15.5:0/4140324851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338039a90 con 0x7f23480ce260 2023-11-24T06:15:30.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.973+0000 7f2336ffd640 1 -- 172.21.15.5:0/4140324851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2338039bf0 con 0x7f23480ce260 2023-11-24T06:15:30.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.973+0000 7f2336ffd640 1 -- 172.21.15.5:0/4140324851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338043410 con 0x7f23480ce260 2023-11-24T06:15:30.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.973+0000 7f234c997640 1 -- 172.21.15.5:0/4140324851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23480cbdc0 con 0x7f23480ce260 2023-11-24T06:15:30.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.974+0000 7f234c997640 1 -- 172.21.15.5:0/4140324851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23480cc380 con 0x7f23480ce260 2023-11-24T06:15:30.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.974+0000 7f2336ffd640 1 -- 172.21.15.5:0/4140324851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2338043870 con 0x7f23480ce260 2023-11-24T06:15:30.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.974+0000 7f2336ffd640 1 -- 172.21.15.5:0/4140324851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f233804c3f0 con 0x7f23480ce260 2023-11-24T06:15:30.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.975+0000 7f234c997640 1 -- 172.21.15.5:0/4140324851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2348074770 con 0x7f23480ce260 2023-11-24T06:15:30.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:30.977+0000 7f2336ffd640 1 -- 172.21.15.5:0/4140324851 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f2338048070 con 0x7f23480ce260 2023-11-24T06:15:31.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:31.043+0000 7f234c997640 1 -- 172.21.15.5:0/4140324851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23480b5ae0 con 0x7f23480ce260 2023-11-24T06:15:31.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:31.044+0000 7f2336ffd640 1 -- 172.21.15.5:0/4140324851 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f2338034070 con 0x7f23480ce260 2023-11-24T06:15:31.046 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:15:31.046 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:30.647672+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:15:31.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:31.045+0000 7f234c997640 1 -- 172.21.15.5:0/4140324851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23480ce260 msgr2=0x7f23480c97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:31.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:31.045+0000 7f234c997640 1 --2- 172.21.15.5:0/4140324851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23480ce260 0x7f23480c97b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2338000c00 tx=0x7f2338004730 comp rx=0 tx=0).stop 2023-11-24T06:15:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:31.045+0000 7f234c997640 1 -- 172.21.15.5:0/4140324851 shutdown_connections 2023-11-24T06:15:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:31.045+0000 7f234c997640 1 --2- 172.21.15.5:0/4140324851 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23480ce6c0 0x7f23480c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:31.045+0000 7f234c997640 1 --2- 172.21.15.5:0/4140324851 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f234815d4d0 0x7f23480c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:31.045+0000 7f234c997640 1 --2- 172.21.15.5:0/4140324851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23480ce260 0x7f23480c97b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:31.045+0000 7f234c997640 1 -- 172.21.15.5:0/4140324851 >> 172.21.15.5:0/4140324851 conn(0x7f2348071f90 msgr2=0x7f23480b62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:31.045+0000 7f234c997640 1 -- 172.21.15.5:0/4140324851 shutdown_connections 2023-11-24T06:15:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:31.045+0000 7f234c997640 1 -- 172.21.15.5:0/4140324851 wait complete. 2023-11-24T06:15:31.062 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-24T06:15:31.062 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-24T06:15:31.394 INFO:tasks.ceph.osd.1.smithi005.stderr:2023-11-24T06:15:31.392+0000 7f9c90e02500 -1 Falling back to public interface 2023-11-24T06:15:31.410 INFO:tasks.ceph.osd.5.smithi006.stderr:2023-11-24T06:15:31.408+0000 7f5839022500 -1 Falling back to public interface 2023-11-24T06:15:31.410 INFO:tasks.ceph.osd.3.smithi005.stderr:2023-11-24T06:15:31.408+0000 7fa57d107500 -1 Falling back to public interface 2023-11-24T06:15:31.427 INFO:tasks.ceph.osd.0.smithi005.stderr:2023-11-24T06:15:31.426+0000 7f63cee8c500 -1 Falling back to public interface 2023-11-24T06:15:31.435 INFO:tasks.ceph.osd.7.smithi006.stderr:2023-11-24T06:15:31.433+0000 7f587560d500 -1 Falling back to public interface 2023-11-24T06:15:31.438 INFO:tasks.ceph.osd.6.smithi006.stderr:2023-11-24T06:15:31.436+0000 7fa485f86500 -1 Falling back to public interface 2023-11-24T06:15:31.445 INFO:tasks.ceph.osd.2.smithi005.stderr:2023-11-24T06:15:31.443+0000 7f216fa12500 -1 Falling back to public interface 2023-11-24T06:15:31.446 INFO:tasks.ceph.osd.4.smithi006.stderr:2023-11-24T06:15:31.445+0000 7fd6d8a13500 -1 Falling back to public interface 2023-11-24T06:15:31.450 INFO:tasks.ceph.osd.6.smithi006.stderr:2023-11-24T06:15:31.449+0000 7fa484f2f640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6823/0: (98) Address already in use 2023-11-24T06:15:31.536 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:31.534+0000 7f3a1d62e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T06:15:31.812 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:31.810+0000 7f1d2dc3b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T06:15:31.869 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:15:31.878 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:31.874+0000 7f3a1d62e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T06:15:31.939 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:31.937+0000 7f3a1d62e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T06:15:32.034 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.032+0000 7f28c96d3640 1 Processor -- start 2023-11-24T06:15:32.035 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.032+0000 7f28c96d3640 1 -- start start 2023-11-24T06:15:32.036 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.034+0000 7f28c96d3640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f28c40ca240 0x7f28c40cb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:32.036 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.034+0000 7f28c96d3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f28c4071ef0 0x7f28c4164ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:32.037 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.034+0000 7f28c96d3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c4070ab0 con 0x7f28c40c9750 2023-11-24T06:15:32.037 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.034+0000 7f28c96d3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c4144370 con 0x7f28c4071ef0 2023-11-24T06:15:32.037 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.034+0000 7f28c96d3640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c40cb790 con 0x7f28c40ca240 2023-11-24T06:15:32.037 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.034+0000 7f28c2ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f28c40c9750 0x7f28c40c9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:58512/0 (socket says 172.21.15.5:58512) 2023-11-24T06:15:32.037 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.034+0000 7f28c2ffd640 1 -- 172.21.15.5:0/160454578 learned_addr learned my addr 172.21.15.5:0/160454578 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:32.037 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.034+0000 7f28c37fe640 1 --2- 172.21.15.5:0/160454578 >> v2:172.21.15.5:3300/0 conn(0x7f28c4071ef0 0x7f28c4164ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:32.037 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.034+0000 7f28c27fc640 1 --2- 172.21.15.5:0/160454578 >> v2:172.21.15.6:3300/0 conn(0x7f28c40ca240 0x7f28c40cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:32.037 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.034+0000 7f28c1ffb640 1 -- 172.21.15.5:0/160454578 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3742868173 0 0) 0x7f28c4070ab0 con 0x7f28c40c9750 2023-11-24T06:15:32.037 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.035+0000 7f28c1ffb640 1 -- 172.21.15.5:0/160454578 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28ac003520 con 0x7f28c40c9750 2023-11-24T06:15:32.038 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.035+0000 7f28c37fe640 1 -- 172.21.15.5:0/160454578 >> v2:172.21.15.6:3300/0 conn(0x7f28c40ca240 msgr2=0x7f28c40cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:32.038 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.035+0000 7f28c37fe640 1 --2- 172.21.15.5:0/160454578 >> v2:172.21.15.6:3300/0 conn(0x7f28c40ca240 0x7f28c40cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:32.038 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.035+0000 7f28c37fe640 1 -- 172.21.15.5:0/160454578 >> v1:172.21.15.5:6789/0 conn(0x7f28c40c9750 legacy=0x7f28c40c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:32.038 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.035+0000 7f28c37fe640 1 -- 172.21.15.5:0/160454578 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c40ca740 con 0x7f28c4071ef0 2023-11-24T06:15:32.041 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.036+0000 7f28c37fe640 1 --2- 172.21.15.5:0/160454578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c4071ef0 0x7f28c4164ce0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f28c4067f80 tx=0x7f28bc02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89a1c3b19011430 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:32.041 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.036+0000 7f28c1ffb640 1 -- 172.21.15.5:0/160454578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc02cbe0 con 0x7f28c4071ef0 2023-11-24T06:15:32.041 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.036+0000 7f28c1ffb640 1 -- 172.21.15.5:0/160454578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc02cd40 con 0x7f28c4071ef0 2023-11-24T06:15:32.041 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.036+0000 7f28c1ffb640 1 -- 172.21.15.5:0/160454578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc034660 con 0x7f28c4071ef0 2023-11-24T06:15:32.042 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.036+0000 7f28c96d3640 1 -- 172.21.15.5:0/160454578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c4071ef0 msgr2=0x7f28c4164ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:32.042 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.037+0000 7f28c96d3640 1 --2- 172.21.15.5:0/160454578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c4071ef0 0x7f28c4164ce0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f28c4067f80 tx=0x7f28bc02be30 comp rx=0 tx=0).stop 2023-11-24T06:15:32.042 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.037+0000 7f28c96d3640 1 -- 172.21.15.5:0/160454578 shutdown_connections 2023-11-24T06:15:32.042 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.037+0000 7f28c96d3640 1 --2- 172.21.15.5:0/160454578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c4071ef0 0x7f28c4164ce0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:32.042 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.037+0000 7f28c96d3640 1 --2- 172.21.15.5:0/160454578 >> v2:172.21.15.6:3300/0 conn(0x7f28c40ca240 0x7f28c40cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:32.043 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.037+0000 7f28c96d3640 1 -- 172.21.15.5:0/160454578 >> 172.21.15.5:0/160454578 conn(0x7f28c40cee90 msgr2=0x7f28c40cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:32.043 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.037+0000 7f28c96d3640 1 -- 172.21.15.5:0/160454578 shutdown_connections 2023-11-24T06:15:32.043 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.037+0000 7f28c96d3640 1 -- 172.21.15.5:0/160454578 wait complete. 2023-11-24T06:15:32.043 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.038+0000 7f28c96d3640 1 Processor -- start 2023-11-24T06:15:32.043 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.039+0000 7f28c96d3640 1 -- start start 2023-11-24T06:15:32.045 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.039+0000 7f28c96d3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c4071ef0 0x7f28c41f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:32.045 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.039+0000 7f28c96d3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28c40c9750 0x7f28c413dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:32.045 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c96d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40ca240 0x7f28c413e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:32.045 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c96d3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c41f4590 con 0x7f28c40ca240 2023-11-24T06:15:32.045 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c96d3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c41f4870 con 0x7f28c4071ef0 2023-11-24T06:15:32.046 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c96d3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c41f4b50 con 0x7f28c40c9750 2023-11-24T06:15:32.046 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40ca240 0x7f28c413e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:32.046 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c2ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c4071ef0 0x7f28c41f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:32.046 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c27fc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28c40c9750 0x7f28c413dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:32.046 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40ca240 0x7f28c413e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46306/0 (socket says 172.21.15.5:46306) 2023-11-24T06:15:32.046 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c37fe640 1 -- 172.21.15.5:0/1280216897 learned_addr learned my addr 172.21.15.5:0/1280216897 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:32.046 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c37fe640 1 -- 172.21.15.5:0/1280216897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28c40c9750 msgr2=0x7f28c413dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:32.046 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c37fe640 1 --2- 172.21.15.5:0/1280216897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28c40c9750 0x7f28c413dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:32.046 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c37fe640 1 -- 172.21.15.5:0/1280216897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c4071ef0 msgr2=0x7f28c41f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:32.046 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c37fe640 1 --2- 172.21.15.5:0/1280216897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c4071ef0 0x7f28c41f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:32.046 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c37fe640 1 -- 172.21.15.5:0/1280216897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc009a70 con 0x7f28c40ca240 2023-11-24T06:15:32.047 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c2ffd640 1 --2- 172.21.15.5:0/1280216897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c4071ef0 0x7f28c41f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:15:32.047 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.040+0000 7f28c27fc640 1 --2- 172.21.15.5:0/1280216897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28c40c9750 0x7f28c413dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:15:32.047 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.041+0000 7f28c37fe640 1 --2- 172.21.15.5:0/1280216897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40ca240 0x7f28c413e2c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009e80 tx=0x7f28bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:32.047 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.041+0000 7f28ab7fe640 1 -- 172.21.15.5:0/1280216897 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc040650 con 0x7f28c40ca240 2023-11-24T06:15:32.047 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.041+0000 7f28ab7fe640 1 -- 172.21.15.5:0/1280216897 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc0407b0 con 0x7f28c40ca240 2023-11-24T06:15:32.047 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.041+0000 7f28c96d3640 1 -- 172.21.15.5:0/1280216897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c413fa10 con 0x7f28c40ca240 2023-11-24T06:15:32.047 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.041+0000 7f28ab7fe640 1 -- 172.21.15.5:0/1280216897 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc040b60 con 0x7f28c40ca240 2023-11-24T06:15:32.047 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.041+0000 7f28c96d3640 1 -- 172.21.15.5:0/1280216897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c41f41f0 con 0x7f28c40ca240 2023-11-24T06:15:32.047 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.042+0000 7f28ab7fe640 1 -- 172.21.15.5:0/1280216897 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f28bc049400 con 0x7f28c40ca240 2023-11-24T06:15:32.047 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.042+0000 7f28ab7fe640 1 -- 172.21.15.5:0/1280216897 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f28bc03ead0 con 0x7f28c40ca240 2023-11-24T06:15:32.047 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.042+0000 7f28c96d3640 1 -- 172.21.15.5:0/1280216897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c413ff30 con 0x7f28c40ca240 2023-11-24T06:15:32.048 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.045+0000 7f28ab7fe640 1 -- 172.21.15.5:0/1280216897 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f28c413ff30 con 0x7f28c40ca240 2023-11-24T06:15:32.063 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:32.061+0000 7f3a1d62e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T06:15:32.073 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.071+0000 7f28ab7fe640 1 -- 172.21.15.5:0/1280216897 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 2) v1 ==== 44567+0+0 (secure 0 0 0) 0x7f28bc03ed30 con 0x7f28c40ca240 2023-11-24T06:15:32.122 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.120+0000 7f28c96d3640 1 -- 172.21.15.5:0/1280216897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28c404e840 con 0x7f28c40ca240 2023-11-24T06:15:32.123 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.121+0000 7f28ab7fe640 1 -- 172.21.15.5:0/1280216897 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f28bc039020 con 0x7f28c40ca240 2023-11-24T06:15:32.123 INFO:teuthology.misc.health.smithi005.stdout: 2023-11-24T06:15:32.123 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":10,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:30.647672+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:15:32.124 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.122+0000 7f28c96d3640 1 -- 172.21.15.5:0/1280216897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40ca240 msgr2=0x7f28c413e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:32.124 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.122+0000 7f28c96d3640 1 --2- 172.21.15.5:0/1280216897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40ca240 0x7f28c413e2c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f28bc009e80 tx=0x7f28bc004a40 comp rx=0 tx=0).stop 2023-11-24T06:15:32.124 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.122+0000 7f28c96d3640 1 -- 172.21.15.5:0/1280216897 shutdown_connections 2023-11-24T06:15:32.124 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.122+0000 7f28c96d3640 1 --2- 172.21.15.5:0/1280216897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28c40c9750 0x7f28c413dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:32.124 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.122+0000 7f28c96d3640 1 --2- 172.21.15.5:0/1280216897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28c40ca240 0x7f28c413e2c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:32.124 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.122+0000 7f28c96d3640 1 --2- 172.21.15.5:0/1280216897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28c4071ef0 0x7f28c41f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:32.124 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.122+0000 7f28c96d3640 1 -- 172.21.15.5:0/1280216897 >> 172.21.15.5:0/1280216897 conn(0x7f28c40cee90 msgr2=0x7f28c40b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:32.125 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.122+0000 7f28c96d3640 1 -- 172.21.15.5:0/1280216897 shutdown_connections 2023-11-24T06:15:32.125 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:32.122+0000 7f28c96d3640 1 -- 172.21.15.5:0/1280216897 wait complete. 2023-11-24T06:15:32.138 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-24T06:15:32.138 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:32.137+0000 7f1d2dc3b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T06:15:32.201 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:32.199+0000 7f1d2dc3b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T06:15:32.319 INFO:tasks.ceph.mgr.x.smithi006.stderr:2023-11-24T06:15:32.317+0000 7f1d2dc3b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T06:15:33.093 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:33.091+0000 7f39be19a640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-24T06:15:33.098 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:15:33.097+0000 7f39be19a640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-24T06:15:34.983 INFO:tasks.ceph.osd.0.smithi005.stderr:2023-11-24T06:15:34.982+0000 7f63cee8c500 -1 osd.0 0 log_to_monitors true 2023-11-24T06:15:35.011 INFO:tasks.ceph.osd.5.smithi006.stderr:2023-11-24T06:15:35.009+0000 7f5839022500 -1 osd.5 0 log_to_monitors true 2023-11-24T06:15:35.029 INFO:tasks.ceph.osd.2.smithi005.stderr:2023-11-24T06:15:35.027+0000 7f216fa12500 -1 osd.2 0 log_to_monitors true 2023-11-24T06:15:35.038 INFO:tasks.ceph.osd.6.smithi006.stderr:2023-11-24T06:15:35.036+0000 7fa485f86500 -1 osd.6 0 log_to_monitors true 2023-11-24T06:15:35.168 INFO:tasks.ceph.osd.1.smithi005.stderr:2023-11-24T06:15:35.166+0000 7f9c90e02500 -1 osd.1 0 log_to_monitors true 2023-11-24T06:15:35.231 INFO:tasks.ceph.osd.3.smithi005.stderr:2023-11-24T06:15:35.229+0000 7fa57d107500 -1 osd.3 0 log_to_monitors true 2023-11-24T06:15:35.273 INFO:tasks.ceph.osd.7.smithi006.stderr:2023-11-24T06:15:35.271+0000 7f587560d500 -1 osd.7 0 log_to_monitors true 2023-11-24T06:15:35.298 INFO:tasks.ceph.osd.4.smithi006.stderr:2023-11-24T06:15:35.296+0000 7fd6d8a13500 -1 osd.4 0 log_to_monitors true 2023-11-24T06:15:36.931 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:15:36.939 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:15:37.443 INFO:tasks.ceph.osd.2.smithi005.stderr:2023-11-24T06:15:37.441+0000 7f216ac6c640 -1 osd.2 0 waiting for initial osdmap 2023-11-24T06:15:37.450 INFO:tasks.ceph.osd.0.smithi005.stderr:2023-11-24T06:15:37.448+0000 7f63ca5f7640 -1 osd.0 0 waiting for initial osdmap 2023-11-24T06:15:37.459 INFO:tasks.ceph.osd.0.smithi005.stderr:2023-11-24T06:15:37.457+0000 7f63c2c30640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:15:37.462 INFO:tasks.ceph.osd.2.smithi005.stderr:2023-11-24T06:15:37.460+0000 7f21632a5640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:15:37.832 INFO:tasks.ceph.osd.6.smithi006.stderr:2023-11-24T06:15:37.830+0000 7fa4816f3640 -1 osd.6 0 waiting for initial osdmap 2023-11-24T06:15:37.847 INFO:tasks.ceph.osd.5.smithi006.stderr:2023-11-24T06:15:37.845+0000 7f5835a05640 -1 osd.5 0 waiting for initial osdmap 2023-11-24T06:15:37.850 INFO:tasks.ceph.osd.6.smithi006.stderr:2023-11-24T06:15:37.849+0000 7fa479d2c640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:15:37.865 INFO:tasks.ceph.osd.5.smithi006.stderr:2023-11-24T06:15:37.863+0000 7f582c806640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:15:38.401 INFO:tasks.ceph.osd.3.smithi005.stderr:2023-11-24T06:15:38.399+0000 7fa57a0ac640 -1 osd.3 0 waiting for initial osdmap 2023-11-24T06:15:38.401 INFO:tasks.ceph.osd.1.smithi005.stderr:2023-11-24T06:15:38.400+0000 7f9c8c56d640 -1 osd.1 0 waiting for initial osdmap 2023-11-24T06:15:38.407 INFO:tasks.ceph.osd.1.smithi005.stderr:2023-11-24T06:15:38.406+0000 7f9c84ba6640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:15:38.412 INFO:tasks.ceph.osd.3.smithi005.stderr:2023-11-24T06:15:38.410+0000 7fa570ead640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:15:38.681 INFO:tasks.ceph.osd.7.smithi006.stderr:2023-11-24T06:15:38.679+0000 7f58707b9640 -1 osd.7 0 waiting for initial osdmap 2023-11-24T06:15:38.681 INFO:tasks.ceph.osd.4.smithi006.stderr:2023-11-24T06:15:38.680+0000 7fd6d3aab640 -1 osd.4 0 waiting for initial osdmap 2023-11-24T06:15:38.689 INFO:tasks.ceph.osd.7.smithi006.stderr:2023-11-24T06:15:38.687+0000 7f5868df2640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:15:38.691 INFO:tasks.ceph.osd.4.smithi006.stderr:2023-11-24T06:15:38.689+0000 7fd6cc0e4640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T06:15:38.943 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:15:39.144 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.142+0000 7fb73f59e640 1 Processor -- start 2023-11-24T06:15:39.144 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.142+0000 7fb73f59e640 1 -- start start 2023-11-24T06:15:39.146 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.144+0000 7fb73f59e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb740071c70 0x7fb74006a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:39.146 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.144+0000 7fb73f59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb7400720d0 0x7fb74006ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:39.146 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.144+0000 7fb73f59e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb740070090 con 0x7fb740161770 2023-11-24T06:15:39.146 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.144+0000 7fb73f59e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb740070370 con 0x7fb7400720d0 2023-11-24T06:15:39.146 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.144+0000 7fb73f59e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb740070650 con 0x7fb740071c70 2023-11-24T06:15:39.146 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.144+0000 7fb73ed9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb7400720d0 0x7fb74006ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:39.146 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.144+0000 7fb73ed9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb7400720d0 0x7fb74006ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37128/0 (socket says 172.21.15.5:37128) 2023-11-24T06:15:39.147 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.144+0000 7fb73ed9d640 1 -- 172.21.15.5:0/368448308 learned_addr learned my addr 172.21.15.5:0/368448308 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:39.147 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.144+0000 7fb73dd9b640 1 --2- 172.21.15.5:0/368448308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb740071c70 0x7fb74006a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:39.147 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.145+0000 7fb73ed9d640 1 -- 172.21.15.5:0/368448308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb740071c70 msgr2=0x7fb74006a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:39.147 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.145+0000 7fb73ed9d640 1 --2- 172.21.15.5:0/368448308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb740071c70 0x7fb74006a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:39.147 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.145+0000 7fb73ed9d640 1 -- 172.21.15.5:0/368448308 >> v1:172.21.15.6:6789/0 conn(0x7fb740161770 legacy=0x7fb740071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:39.147 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.145+0000 7fb73ed9d640 1 -- 172.21.15.5:0/368448308 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb740070930 con 0x7fb7400720d0 2023-11-24T06:15:39.148 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.146+0000 7fb73ed9d640 1 --2- 172.21.15.5:0/368448308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7400720d0 0x7fb74006ada0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb738009e50 tx=0x7fb73802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1795c68c75e5a3f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:39.148 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.146+0000 7fb73d59a640 1 -- 172.21.15.5:0/368448308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73802ca70 con 0x7fb7400720d0 2023-11-24T06:15:39.148 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.146+0000 7fb73d59a640 1 -- 172.21.15.5:0/368448308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73802cbd0 con 0x7fb7400720d0 2023-11-24T06:15:39.148 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.146+0000 7fb73d59a640 1 -- 172.21.15.5:0/368448308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7380343f0 con 0x7fb7400720d0 2023-11-24T06:15:39.149 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.147+0000 7fb73f59e640 1 -- 172.21.15.5:0/368448308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7400720d0 msgr2=0x7fb74006ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:39.149 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.147+0000 7fb73f59e640 1 --2- 172.21.15.5:0/368448308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7400720d0 0x7fb74006ada0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb738009e50 tx=0x7fb73802bed0 comp rx=0 tx=0).stop 2023-11-24T06:15:39.149 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.147+0000 7fb73f59e640 1 -- 172.21.15.5:0/368448308 shutdown_connections 2023-11-24T06:15:39.149 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.147+0000 7fb73f59e640 1 --2- 172.21.15.5:0/368448308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7400720d0 0x7fb74006ada0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:39.149 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.147+0000 7fb73f59e640 1 --2- 172.21.15.5:0/368448308 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb740071c70 0x7fb74006a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:39.149 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.147+0000 7fb73f59e640 1 -- 172.21.15.5:0/368448308 >> 172.21.15.5:0/368448308 conn(0x7fb7400b0d00 msgr2=0x7fb7400b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:39.150 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.147+0000 7fb73f59e640 1 -- 172.21.15.5:0/368448308 shutdown_connections 2023-11-24T06:15:39.150 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.147+0000 7fb73f59e640 1 -- 172.21.15.5:0/368448308 wait complete. 2023-11-24T06:15:39.151 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.149+0000 7fb73f59e640 1 Processor -- start 2023-11-24T06:15:39.151 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.149+0000 7fb73f59e640 1 -- start start 2023-11-24T06:15:39.153 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.150+0000 7fb73f59e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb740071c70 0x7fb74015b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:39.153 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.150+0000 7fb73f59e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7400720d0 0x7fb7401fa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:39.153 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.150+0000 7fb73f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7400cb360 0x7fb740145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:39.153 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.151+0000 7fb73f59e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74015ae30 con 0x7fb7400cb360 2023-11-24T06:15:39.154 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.151+0000 7fb73f59e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400cb810 con 0x7fb740071c70 2023-11-24T06:15:39.154 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.151+0000 7fb73f59e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400cba60 con 0x7fb7400720d0 2023-11-24T06:15:39.154 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.151+0000 7fb73ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7400cb360 0x7fb740145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:39.154 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.151+0000 7fb73ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7400cb360 0x7fb740145c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37144/0 (socket says 172.21.15.5:37144) 2023-11-24T06:15:39.154 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.151+0000 7fb73ed9d640 1 -- 172.21.15.5:0/4128540791 learned_addr learned my addr 172.21.15.5:0/4128540791 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:39.154 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.151+0000 7fb73e59c640 1 --2- 172.21.15.5:0/4128540791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb740071c70 0x7fb74015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:39.154 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.151+0000 7fb73dd9b640 1 --2- 172.21.15.5:0/4128540791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7400720d0 0x7fb7401fa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:39.154 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.151+0000 7fb73ed9d640 1 -- 172.21.15.5:0/4128540791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7400720d0 msgr2=0x7fb7401fa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:39.154 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.151+0000 7fb73ed9d640 1 --2- 172.21.15.5:0/4128540791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7400720d0 0x7fb7401fa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:39.154 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.151+0000 7fb73ed9d640 1 -- 172.21.15.5:0/4128540791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb740071c70 msgr2=0x7fb74015b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:39.154 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.151+0000 7fb73ed9d640 1 --2- 172.21.15.5:0/4128540791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb740071c70 0x7fb74015b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:39.154 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.151+0000 7fb73ed9d640 1 -- 172.21.15.5:0/4128540791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb738009b30 con 0x7fb7400cb360 2023-11-24T06:15:39.155 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.152+0000 7fb73ed9d640 1 --2- 172.21.15.5:0/4128540791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7400cb360 0x7fb740145c60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb738036040 tx=0x7fb738004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:39.155 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.152+0000 7fb72effd640 1 -- 172.21.15.5:0/4128540791 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738009510 con 0x7fb7400cb360 2023-11-24T06:15:39.155 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.152+0000 7fb72effd640 1 -- 172.21.15.5:0/4128540791 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73803e450 con 0x7fb7400cb360 2023-11-24T06:15:39.156 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.152+0000 7fb72effd640 1 -- 172.21.15.5:0/4128540791 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73803e7e0 con 0x7fb7400cb360 2023-11-24T06:15:39.156 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.152+0000 7fb73f59e640 1 -- 172.21.15.5:0/4128540791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb74015a490 con 0x7fb7400cb360 2023-11-24T06:15:39.156 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.152+0000 7fb73f59e640 1 -- 172.21.15.5:0/4128540791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb74015aa50 con 0x7fb7400cb360 2023-11-24T06:15:39.156 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.153+0000 7fb73f59e640 1 -- 172.21.15.5:0/4128540791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74015af70 con 0x7fb7400cb360 2023-11-24T06:15:39.156 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.154+0000 7fb72effd640 1 -- 172.21.15.5:0/4128540791 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7fb73803e940 con 0x7fb7400cb360 2023-11-24T06:15:39.157 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.155+0000 7fb72effd640 1 --2- 172.21.15.5:0/4128540791 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb72006adb0 0x7fb72006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:39.157 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.155+0000 7fb72effd640 1 -- 172.21.15.5:0/4128540791 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3911+0+0 (secure 0 0 0) 0x7fb7380abbd0 con 0x7fb7400cb360 2023-11-24T06:15:39.157 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.155+0000 7fb73e59c640 1 --2- 172.21.15.5:0/4128540791 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb72006adb0 0x7fb72006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:39.158 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.156+0000 7fb73e59c640 1 --2- 172.21.15.5:0/4128540791 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb72006adb0 0x7fb72006d270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb730006fd0 tx=0x7fb730008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:39.162 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.160+0000 7fb72effd640 1 -- 172.21.15.5:0/4128540791 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb74015af70 con 0x7fb7400cb360 2023-11-24T06:15:39.349 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.347+0000 7fb73f59e640 1 -- 172.21.15.5:0/4128540791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb74015af70 con 0x7fb7400cb360 2023-11-24T06:15:39.350 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.348+0000 7fb72effd640 1 -- 172.21.15.5:0/4128540791 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+9840 (secure 0 0 0) 0x7fb74015af70 con 0x7fb7400cb360 2023-11-24T06:15:39.350 INFO:teuthology.misc.health.smithi005.stdout: 2023-11-24T06:15:39.350 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":14,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:38.459016+0000","last_up_change":"2023-11-24T06:15:38.459016+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:15:39.352 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.351+0000 7fb73f59e640 1 -- 172.21.15.5:0/4128540791 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb72006adb0 msgr2=0x7fb72006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:39.353 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.351+0000 7fb73f59e640 1 --2- 172.21.15.5:0/4128540791 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb72006adb0 0x7fb72006d270 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb730006fd0 tx=0x7fb730008040 comp rx=0 tx=0).stop 2023-11-24T06:15:39.353 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.351+0000 7fb73f59e640 1 -- 172.21.15.5:0/4128540791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7400cb360 msgr2=0x7fb740145c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:39.353 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.351+0000 7fb73f59e640 1 --2- 172.21.15.5:0/4128540791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7400cb360 0x7fb740145c60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb738036040 tx=0x7fb738004a40 comp rx=0 tx=0).stop 2023-11-24T06:15:39.353 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.351+0000 7fb73f59e640 1 -- 172.21.15.5:0/4128540791 shutdown_connections 2023-11-24T06:15:39.353 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.351+0000 7fb73f59e640 1 --2- 172.21.15.5:0/4128540791 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb72006adb0 0x7fb72006d270 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:39.353 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.351+0000 7fb73f59e640 1 --2- 172.21.15.5:0/4128540791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb7400720d0 0x7fb7401fa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:39.354 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.351+0000 7fb73f59e640 1 --2- 172.21.15.5:0/4128540791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7400cb360 0x7fb740145c60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:39.354 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.351+0000 7fb73f59e640 1 --2- 172.21.15.5:0/4128540791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb740071c70 0x7fb74015b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:39.354 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.351+0000 7fb73f59e640 1 -- 172.21.15.5:0/4128540791 >> 172.21.15.5:0/4128540791 conn(0x7fb7400b0d00 msgr2=0x7fb74015fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:39.354 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.351+0000 7fb73f59e640 1 -- 172.21.15.5:0/4128540791 shutdown_connections 2023-11-24T06:15:39.354 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:39.351+0000 7fb73f59e640 1 -- 172.21.15.5:0/4128540791 wait complete. 2023-11-24T06:15:39.366 DEBUG:teuthology.misc:6 of 8 OSDs are up 2023-11-24T06:15:46.172 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T06:15:46.354 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.350+0000 7ff5e2306640 1 Processor -- start 2023-11-24T06:15:46.354 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.351+0000 7ff5e2306640 1 -- start start 2023-11-24T06:15:46.355 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.352+0000 7ff5e2306640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5dc15d320 0x7ff5dc0d14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.355 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.352+0000 7ff5e2306640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff5dc15d700 0x7ff5dc0d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.355 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.352+0000 7ff5e2306640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5dc0afc60 con 0x7ff5dc0b2060 2023-11-24T06:15:46.355 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.352+0000 7ff5e2306640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5dc0d3ef0 con 0x7ff5dc15d700 2023-11-24T06:15:46.355 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.352+0000 7ff5e2306640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0d0e80 con 0x7ff5dc15d320 2023-11-24T06:15:46.357 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.352+0000 7ff5e1b05640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff5dc0b2060 0x7ff5dc0b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48010/0 (socket says 172.21.15.5:48010) 2023-11-24T06:15:46.357 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.352+0000 7ff5e1b05640 1 -- 172.21.15.5:0/4022567559 learned_addr learned my addr 172.21.15.5:0/4022567559 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:46.357 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5e0b03640 1 --2- 172.21.15.5:0/4022567559 >> v2:172.21.15.6:3300/0 conn(0x7ff5dc15d700 0x7ff5dc0d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.358 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5e1304640 1 --2- 172.21.15.5:0/4022567559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5dc15d320 0x7ff5dc0d14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.358 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 -- 172.21.15.5:0/4022567559 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285665021 0 0) 0x7ff5dc0afc60 con 0x7ff5dc0b2060 2023-11-24T06:15:46.358 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 -- 172.21.15.5:0/4022567559 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5c8003520 con 0x7ff5dc0b2060 2023-11-24T06:15:46.358 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 -- 172.21.15.5:0/4022567559 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4089088743 0 0) 0x7ff5c8003520 con 0x7ff5dc0b2060 2023-11-24T06:15:46.358 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 -- 172.21.15.5:0/4022567559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5dc15d320 msgr2=0x7ff5dc0d14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:46.358 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 --2- 172.21.15.5:0/4022567559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5dc15d320 0x7ff5dc0d14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.358 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 -- 172.21.15.5:0/4022567559 >> v2:172.21.15.6:3300/0 conn(0x7ff5dc15d700 msgr2=0x7ff5dc0d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:46.358 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 --2- 172.21.15.5:0/4022567559 >> v2:172.21.15.6:3300/0 conn(0x7ff5dc15d700 0x7ff5dc0d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.358 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 -- 172.21.15.5:0/4022567559 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc0d1160 con 0x7ff5dc0b2060 2023-11-24T06:15:46.358 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 -- 172.21.15.5:0/4022567559 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7ff5d40019f0 con 0x7ff5dc0b2060 2023-11-24T06:15:46.358 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 -- 172.21.15.5:0/4022567559 >> v1:172.21.15.5:6789/0 conn(0x7ff5dc0b2060 legacy=0x7ff5dc0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:46.359 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 --2- 172.21.15.5:0/4022567559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c8003970 0x7ff5c8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.359 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 --2- 172.21.15.5:0/4022567559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5dc15d320 0x7ff5c80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.359 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 --2- 172.21.15.5:0/4022567559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5c8004d00 0x7ff5c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.359 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 -- 172.21.15.5:0/4022567559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0d3ef0 con 0x7ff5c8003970 2023-11-24T06:15:46.359 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 -- 172.21.15.5:0/4022567559 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0d0e80 con 0x7ff5c8004d00 2023-11-24T06:15:46.359 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.353+0000 7ff5d3fff640 1 -- 172.21.15.5:0/4022567559 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c800b980 con 0x7ff5dc15d320 2023-11-24T06:15:46.359 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.354+0000 7ff5e1304640 1 --2- 172.21.15.5:0/4022567559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c8003970 0x7ff5c8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.359 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.354+0000 7ff5e0b03640 1 --2- 172.21.15.5:0/4022567559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5dc15d320 0x7ff5c80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.359 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.354+0000 7ff5e1b05640 1 --2- 172.21.15.5:0/4022567559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5c8004d00 0x7ff5c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.359 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.354+0000 7ff5e1304640 1 -- 172.21.15.5:0/4022567559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5dc15d320 msgr2=0x7ff5c80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:46.359 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.354+0000 7ff5e1304640 1 --2- 172.21.15.5:0/4022567559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5dc15d320 0x7ff5c80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.359 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.354+0000 7ff5e1304640 1 -- 172.21.15.5:0/4022567559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5c8004d00 msgr2=0x7ff5c8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:46.360 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.354+0000 7ff5e1304640 1 --2- 172.21.15.5:0/4022567559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5c8004d00 0x7ff5c8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.360 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.354+0000 7ff5e1304640 1 -- 172.21.15.5:0/4022567559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c8003520 con 0x7ff5c8003970 2023-11-24T06:15:46.360 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.354+0000 7ff5e0b03640 1 --2- 172.21.15.5:0/4022567559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5dc15d320 0x7ff5c80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:15:46.360 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.354+0000 7ff5e1b05640 1 --2- 172.21.15.5:0/4022567559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5c8004d00 0x7ff5c8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:15:46.361 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.355+0000 7ff5e1304640 1 --2- 172.21.15.5:0/4022567559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c8003970 0x7ff5c8003d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009ac0 tx=0x7ff5c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d408d0214d10b82b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:46.361 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.355+0000 7ff5d3fff640 1 -- 172.21.15.5:0/4022567559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d802c6b0 con 0x7ff5c8003970 2023-11-24T06:15:46.361 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.355+0000 7ff5d3fff640 1 -- 172.21.15.5:0/4022567559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d802c810 con 0x7ff5c8003970 2023-11-24T06:15:46.361 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.355+0000 7ff5d3fff640 1 -- 172.21.15.5:0/4022567559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d802cba0 con 0x7ff5c8003970 2023-11-24T06:15:46.362 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.356+0000 7ff5e2306640 1 -- 172.21.15.5:0/4022567559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c8003970 msgr2=0x7ff5c8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:46.362 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.356+0000 7ff5e2306640 1 --2- 172.21.15.5:0/4022567559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c8003970 0x7ff5c8003d90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8009ac0 tx=0x7ff5c80057f0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.363 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.356+0000 7ff5e1b05640 1 -- 172.21.15.5:0/4022567559 reap_dead start 2023-11-24T06:15:46.363 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.356+0000 7ff5e2306640 1 -- 172.21.15.5:0/4022567559 shutdown_connections 2023-11-24T06:15:46.363 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.356+0000 7ff5e2306640 1 -- 172.21.15.5:0/4022567559 >> 172.21.15.5:0/4022567559 conn(0x7ff5dc0d30a0 msgr2=0x7ff5dc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:46.363 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.356+0000 7ff5e2306640 1 -- 172.21.15.5:0/4022567559 shutdown_connections 2023-11-24T06:15:46.363 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.356+0000 7ff5e2306640 1 -- 172.21.15.5:0/4022567559 wait complete. 2023-11-24T06:15:46.363 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.358+0000 7ff5e2306640 1 Processor -- start 2023-11-24T06:15:46.363 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.358+0000 7ff5e2306640 1 -- start start 2023-11-24T06:15:46.365 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.359+0000 7ff5e2306640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c8003970 0x7ff5dc0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.365 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.359+0000 7ff5e2306640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5dc15b3a0 0x7ff5dc0b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.365 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.359+0000 7ff5e2306640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5dc152020 0x7ff5dc1524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.365 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.359+0000 7ff5e2306640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc1529e0 con 0x7ff5c8003970 2023-11-24T06:15:46.365 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.359+0000 7ff5e2306640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0cd030 con 0x7ff5dc152020 2023-11-24T06:15:46.365 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.359+0000 7ff5e2306640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5dc0cb5e0 con 0x7ff5dc15b3a0 2023-11-24T06:15:46.365 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.359+0000 7ff5e1304640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c8003970 0x7ff5dc0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.365 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.359+0000 7ff5e1304640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c8003970 0x7ff5dc0cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57396/0 (socket says 172.21.15.5:57396) 2023-11-24T06:15:46.366 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.359+0000 7ff5e1304640 1 -- 172.21.15.5:0/2534559161 learned_addr learned my addr 172.21.15.5:0/2534559161 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:46.366 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.360+0000 7ff5e0b03640 1 --2- 172.21.15.5:0/2534559161 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5dc15b3a0 0x7ff5dc0b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.366 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.360+0000 7ff5e1b05640 1 --2- 172.21.15.5:0/2534559161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5dc152020 0x7ff5dc1524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.366 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.360+0000 7ff5e1304640 1 -- 172.21.15.5:0/2534559161 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5dc15b3a0 msgr2=0x7ff5dc0b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:46.366 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.360+0000 7ff5e1304640 1 --2- 172.21.15.5:0/2534559161 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5dc15b3a0 0x7ff5dc0b7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.366 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.360+0000 7ff5e1304640 1 -- 172.21.15.5:0/2534559161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5dc152020 msgr2=0x7ff5dc1524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:46.366 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.360+0000 7ff5e1304640 1 --2- 172.21.15.5:0/2534559161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5dc152020 0x7ff5dc1524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.366 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.360+0000 7ff5e1304640 1 -- 172.21.15.5:0/2534559161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5dc15bb80 con 0x7ff5c8003970 2023-11-24T06:15:46.366 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.360+0000 7ff5e1304640 1 --2- 172.21.15.5:0/2534559161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c8003970 0x7ff5dc0cc9a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8037040 tx=0x7ff5d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:46.366 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.360+0000 7ff5d1ffb640 1 -- 172.21.15.5:0/2534559161 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8003810 con 0x7ff5c8003970 2023-11-24T06:15:46.366 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.360+0000 7ff5d1ffb640 1 -- 172.21.15.5:0/2534559161 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d8003970 con 0x7ff5c8003970 2023-11-24T06:15:46.366 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.361+0000 7ff5d1ffb640 1 -- 172.21.15.5:0/2534559161 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8003ce0 con 0x7ff5c8003970 2023-11-24T06:15:46.367 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.361+0000 7ff5e2306640 1 -- 172.21.15.5:0/2534559161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5dc0cacb0 con 0x7ff5c8003970 2023-11-24T06:15:46.367 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.361+0000 7ff5e2306640 1 -- 172.21.15.5:0/2534559161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5dc0cb270 con 0x7ff5c8003970 2023-11-24T06:15:46.367 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.362+0000 7ff5e2306640 1 -- 172.21.15.5:0/2534559161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5dc0d19d0 con 0x7ff5c8003970 2023-11-24T06:15:46.368 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.363+0000 7ff5d1ffb640 1 -- 172.21.15.5:0/2534559161 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff5d8047450 con 0x7ff5c8003970 2023-11-24T06:15:46.368 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.364+0000 7ff5d1ffb640 1 --2- 172.21.15.5:0/2534559161 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff5c006ae40 0x7ff5c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.368 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.364+0000 7ff5d1ffb640 1 -- 172.21.15.5:0/2534559161 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff5d80ab5f0 con 0x7ff5c8003970 2023-11-24T06:15:46.368 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.364+0000 7ff5e0b03640 1 --2- 172.21.15.5:0/2534559161 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff5c006ae40 0x7ff5c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.368 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.365+0000 7ff5e0b03640 1 --2- 172.21.15.5:0/2534559161 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff5c006ae40 0x7ff5c006d300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0b7360 tx=0x7ff5cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:46.374 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.369+0000 7ff5d1ffb640 1 -- 172.21.15.5:0/2534559161 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff5d8076a90 con 0x7ff5c8003970 2023-11-24T06:15:46.564 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.561+0000 7ff5e2306640 1 -- 172.21.15.5:0/2534559161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5dc04e840 con 0x7ff5c8003970 2023-11-24T06:15:46.566 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.563+0000 7ff5d1ffb640 1 -- 172.21.15.5:0/2534559161 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12706 (secure 0 0 0) 0x7ff5d80044e0 con 0x7ff5c8003970 2023-11-24T06:15:46.566 INFO:teuthology.misc.health.smithi005.stdout: 2023-11-24T06:15:46.567 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":18,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:42.775578+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:15:46.569 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.566+0000 7ff5e2306640 1 -- 172.21.15.5:0/2534559161 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff5c006ae40 msgr2=0x7ff5c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:46.569 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.566+0000 7ff5e2306640 1 --2- 172.21.15.5:0/2534559161 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff5c006ae40 0x7ff5c006d300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc0b7360 tx=0x7ff5cc008040 comp rx=0 tx=0).stop 2023-11-24T06:15:46.569 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.566+0000 7ff5e2306640 1 -- 172.21.15.5:0/2534559161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c8003970 msgr2=0x7ff5dc0cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:46.570 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.566+0000 7ff5e2306640 1 --2- 172.21.15.5:0/2534559161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c8003970 0x7ff5dc0cc9a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8037040 tx=0x7ff5d8004730 comp rx=0 tx=0).stop 2023-11-24T06:15:46.570 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.566+0000 7ff5e2306640 1 -- 172.21.15.5:0/2534559161 shutdown_connections 2023-11-24T06:15:46.570 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.566+0000 7ff5e2306640 1 --2- 172.21.15.5:0/2534559161 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff5c006ae40 0x7ff5c006d300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.570 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.566+0000 7ff5e2306640 1 --2- 172.21.15.5:0/2534559161 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5dc15b3a0 0x7ff5dc0b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.570 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.566+0000 7ff5e2306640 1 --2- 172.21.15.5:0/2534559161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5dc152020 0x7ff5dc1524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.570 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.566+0000 7ff5e2306640 1 --2- 172.21.15.5:0/2534559161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c8003970 0x7ff5dc0cc9a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.570 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.566+0000 7ff5e2306640 1 -- 172.21.15.5:0/2534559161 >> 172.21.15.5:0/2534559161 conn(0x7ff5dc0d30a0 msgr2=0x7ff5dc15db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:46.570 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.566+0000 7ff5e2306640 1 -- 172.21.15.5:0/2534559161 shutdown_connections 2023-11-24T06:15:46.570 INFO:teuthology.misc.health.smithi005.stderr:2023-11-24T06:15:46.566+0000 7ff5e2306640 1 -- 172.21.15.5:0/2534559161 wait complete. 2023-11-24T06:15:46.584 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-24T06:15:46.584 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-24T06:15:46.584 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:15:46.585 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:15:46.589 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:15:46.590 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:15:46.590 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:15:46.595 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:15:46.595 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:15:46.595 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:15:46.604 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:15:46.604 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:15:46.605 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:15:46.614 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:15:46.614 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-24T06:15:46.615 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:15:46.748 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2023-11-24T06:15:46.748 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2023-11-24T06:15:46.751 INFO:tasks.ceph.mds.a.smithi005.stderr:2023-11-24T06:15:46.748+0000 7f71252bd640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-24T06:15:46.776 INFO:tasks.ceph.mds.b.smithi006.stdout:starting mds.b at 2023-11-24T06:15:46.789 INFO:tasks.ceph.mds.d.smithi006.stdout:starting mds.d at 2023-11-24T06:15:46.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.805+0000 7fd0e1415640 1 Processor -- start 2023-11-24T06:15:46.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.805+0000 7fd0e1415640 1 -- start start 2023-11-24T06:15:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.807+0000 7fd0e1415640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd0dc15d550 0x7fd0dc099a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.807+0000 7fd0e1415640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0dc06ab50 0x7fd0dc0b3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.807+0000 7fd0e1415640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0dc0af800 con 0x7fd0dc0af3c0 2023-11-24T06:15:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.807+0000 7fd0e1415640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0dc0afac0 con 0x7fd0dc06ab50 2023-11-24T06:15:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.807+0000 7fd0e1415640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0dc0ae600 con 0x7fd0dc15d550 2023-11-24T06:15:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.807+0000 7fd0d3fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd0dc0af3c0 0x7fd0dc06a440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48016/0 (socket says 172.21.15.5:48016) 2023-11-24T06:15:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.807+0000 7fd0daffd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd0dc15d550 0x7fd0dc099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.807+0000 7fd0d3fff640 1 -- 172.21.15.5:0/2345844548 learned_addr learned my addr 172.21.15.5:0/2345844548 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:46.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.807+0000 7fd0db7fe640 1 --2- 172.21.15.5:0/2345844548 >> v2:172.21.15.5:3300/0 conn(0x7fd0dc06ab50 0x7fd0dc0b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.807+0000 7fd0daffd640 1 -- 172.21.15.5:0/2345844548 >> v2:172.21.15.5:3300/0 conn(0x7fd0dc06ab50 msgr2=0x7fd0dc0b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.807+0000 7fd0daffd640 1 --2- 172.21.15.5:0/2345844548 >> v2:172.21.15.5:3300/0 conn(0x7fd0dc06ab50 0x7fd0dc0b3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.807+0000 7fd0daffd640 1 -- 172.21.15.5:0/2345844548 >> v1:172.21.15.5:6789/0 conn(0x7fd0dc0af3c0 legacy=0x7fd0dc06a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.807+0000 7fd0daffd640 1 -- 172.21.15.5:0/2345844548 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc0ae8e0 con 0x7fd0dc15d550 2023-11-24T06:15:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.807+0000 7fd0da7fc640 1 -- 172.21.15.5:0/2345844548 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827058783 0 0) 0x7fd0dc0af800 con 0x7fd0dc0af3c0 2023-11-24T06:15:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.809+0000 7fd0daffd640 1 --2- 172.21.15.5:0/2345844548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0dc15d550 0x7fd0dc099a00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc067190 tx=0x7fd0cc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d89f0041fb20ec1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.809+0000 7fd0da7fc640 1 -- 172.21.15.5:0/2345844548 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc030610 con 0x7fd0dc15d550 2023-11-24T06:15:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.809+0000 7fd0da7fc640 1 -- 172.21.15.5:0/2345844548 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc030770 con 0x7fd0dc15d550 2023-11-24T06:15:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.809+0000 7fd0da7fc640 1 -- 172.21.15.5:0/2345844548 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc030b00 con 0x7fd0dc15d550 2023-11-24T06:15:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.809+0000 7fd0e1415640 1 -- 172.21.15.5:0/2345844548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0dc15d550 msgr2=0x7fd0dc099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.809+0000 7fd0e1415640 1 --2- 172.21.15.5:0/2345844548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0dc15d550 0x7fd0dc099a00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc067190 tx=0x7fd0cc02b960 comp rx=0 tx=0).stop 2023-11-24T06:15:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.810+0000 7fd0e1415640 1 -- 172.21.15.5:0/2345844548 shutdown_connections 2023-11-24T06:15:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.810+0000 7fd0e1415640 1 --2- 172.21.15.5:0/2345844548 >> v2:172.21.15.5:3300/0 conn(0x7fd0dc06ab50 0x7fd0dc0b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.810+0000 7fd0e1415640 1 --2- 172.21.15.5:0/2345844548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0dc15d550 0x7fd0dc099a00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.810+0000 7fd0e1415640 1 -- 172.21.15.5:0/2345844548 >> 172.21.15.5:0/2345844548 conn(0x7fd0dc071aa0 msgr2=0x7fd0dc0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.810+0000 7fd0e1415640 1 -- 172.21.15.5:0/2345844548 shutdown_connections 2023-11-24T06:15:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.810+0000 7fd0e1415640 1 -- 172.21.15.5:0/2345844548 wait complete. 2023-11-24T06:15:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.811+0000 7fd0e1415640 1 Processor -- start 2023-11-24T06:15:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.812+0000 7fd0e1415640 1 -- start start 2023-11-24T06:15:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0e1415640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0dc06ab50 0x7fd0dc143900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0e1415640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0dc0af3c0 0x7fd0dc150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0e1415640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0dc15d550 0x7fd0dc1530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0e1415640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc151710 con 0x7fd0dc15d550 2023-11-24T06:15:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0e1415640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc1519f0 con 0x7fd0dc0af3c0 2023-11-24T06:15:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0e1415640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0dc151cd0 con 0x7fd0dc06ab50 2023-11-24T06:15:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0db7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0dc15d550 0x7fd0dc1530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0db7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0dc15d550 0x7fd0dc1530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57424/0 (socket says 172.21.15.5:57424) 2023-11-24T06:15:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0db7fe640 1 -- 172.21.15.5:0/2673295524 learned_addr learned my addr 172.21.15.5:0/2673295524 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0d3fff640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0dc0af3c0 0x7fd0dc150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0daffd640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0dc06ab50 0x7fd0dc143900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0db7fe640 1 -- 172.21.15.5:0/2673295524 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0dc06ab50 msgr2=0x7fd0dc143900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0db7fe640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0dc06ab50 0x7fd0dc143900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0db7fe640 1 -- 172.21.15.5:0/2673295524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0dc0af3c0 msgr2=0x7fd0dc150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0db7fe640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0dc0af3c0 0x7fd0dc150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0db7fe640 1 -- 172.21.15.5:0/2673295524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc009400 con 0x7fd0dc15d550 2023-11-24T06:15:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.813+0000 7fd0daffd640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0dc06ab50 0x7fd0dc143900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:15:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.814+0000 7fd0db7fe640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0dc15d550 0x7fd0dc1530c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400ef90 tx=0x7fd0d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.814+0000 7fd0d37fe640 1 -- 172.21.15.5:0/2673295524 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d4009ca0 con 0x7fd0dc15d550 2023-11-24T06:15:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.814+0000 7fd0d37fe640 1 -- 172.21.15.5:0/2673295524 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d400f040 con 0x7fd0dc15d550 2023-11-24T06:15:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.814+0000 7fd0e1415640 1 -- 172.21.15.5:0/2673295524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0dc151fb0 con 0x7fd0dc15d550 2023-11-24T06:15:46.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.814+0000 7fd0d37fe640 1 -- 172.21.15.5:0/2673295524 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d40146c0 con 0x7fd0dc15d550 2023-11-24T06:15:46.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.814+0000 7fd0e1415640 1 -- 172.21.15.5:0/2673295524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0dc143e40 con 0x7fd0dc15d550 2023-11-24T06:15:46.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.815+0000 7fd0e1415640 1 -- 172.21.15.5:0/2673295524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0dc1524d0 con 0x7fd0dc15d550 2023-11-24T06:15:46.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.816+0000 7fd0d37fe640 1 -- 172.21.15.5:0/2673295524 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd0d4014820 con 0x7fd0dc15d550 2023-11-24T06:15:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.817+0000 7fd0d37fe640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd0bc06ae40 0x7fd0bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.817+0000 7fd0d37fe640 1 -- 172.21.15.5:0/2673295524 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd0d408a180 con 0x7fd0dc15d550 2023-11-24T06:15:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.817+0000 7fd0daffd640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd0bc06ae40 0x7fd0bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:46.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.818+0000 7fd0daffd640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd0bc06ae40 0x7fd0bc06d300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc036000 tx=0x7fd0cc03b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:46.823+0000 7fd0d37fe640 1 -- 172.21.15.5:0/2673295524 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0dc1524d0 con 0x7fd0dc15d550 2023-11-24T06:15:47.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.019+0000 7fd0e1415640 1 -- 172.21.15.5:0/2673295524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0dc1524d0 con 0x7fd0dc15d550 2023-11-24T06:15:47.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.021+0000 7fd0d37fe640 1 -- 172.21.15.5:0/2673295524 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12706 (secure 0 0 0) 0x7fd0dc1524d0 con 0x7fd0dc15d550 2023-11-24T06:15:47.024 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:15:47.025 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":18,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:42.775578+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:15:47.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.024+0000 7fd0e1415640 1 -- 172.21.15.5:0/2673295524 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd0bc06ae40 msgr2=0x7fd0bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.024+0000 7fd0e1415640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd0bc06ae40 0x7fd0bc06d300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc036000 tx=0x7fd0cc03b040 comp rx=0 tx=0).stop 2023-11-24T06:15:47.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.024+0000 7fd0e1415640 1 -- 172.21.15.5:0/2673295524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0dc15d550 msgr2=0x7fd0dc1530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.024+0000 7fd0e1415640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0dc15d550 0x7fd0dc1530c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400ef90 tx=0x7fd0d400c490 comp rx=0 tx=0).stop 2023-11-24T06:15:47.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.024+0000 7fd0e1415640 1 -- 172.21.15.5:0/2673295524 shutdown_connections 2023-11-24T06:15:47.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.024+0000 7fd0e1415640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd0bc06ae40 0x7fd0bc06d300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.024+0000 7fd0e1415640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0dc15d550 0x7fd0dc1530c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.024+0000 7fd0e1415640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0dc0af3c0 0x7fd0dc150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.024+0000 7fd0e1415640 1 --2- 172.21.15.5:0/2673295524 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0dc06ab50 0x7fd0dc143900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.024+0000 7fd0e1415640 1 -- 172.21.15.5:0/2673295524 >> 172.21.15.5:0/2673295524 conn(0x7fd0dc071aa0 msgr2=0x7fd0dc15c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:47.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.025+0000 7fd0e1415640 1 -- 172.21.15.5:0/2673295524 shutdown_connections 2023-11-24T06:15:47.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.025+0000 7fd0e1415640 1 -- 172.21.15.5:0/2673295524 wait complete. 2023-11-24T06:15:47.046 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:15:47.046 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:15:47.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.278+0000 7f8bbcf9a640 1 Processor -- start 2023-11-24T06:15:47.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.279+0000 7f8bbcf9a640 1 -- start start 2023-11-24T06:15:47.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.280+0000 7f8bbcf9a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8bb815f270 0x7f8bb80cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:47.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.280+0000 7f8bbcf9a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb815f930 con 0x7f8bb806d9e0 2023-11-24T06:15:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.281+0000 7f8bbcf9a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb80b3b10 con 0x7f8bb8144770 2023-11-24T06:15:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.281+0000 7f8bbcf9a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb80b43b0 con 0x7f8bb815f270 2023-11-24T06:15:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.281+0000 7f8bb7fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8bb806d9e0 0x7f8bb80cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48024/0 (socket says 172.21.15.5:48024) 2023-11-24T06:15:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.281+0000 7f8bb7fff640 1 -- 172.21.15.5:0/2457923829 learned_addr learned my addr 172.21.15.5:0/2457923829 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.281+0000 7f8baffff640 1 --2- 172.21.15.5:0/2457923829 >> v2:172.21.15.6:3300/0 conn(0x7f8bb815f270 0x7f8bb80cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.281+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2457923829 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576334081 0 0) 0x7f8bb815f930 con 0x7f8bb806d9e0 2023-11-24T06:15:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.281+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2457923829 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ba4003520 con 0x7f8bb806d9e0 2023-11-24T06:15:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2457923829 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3727660707 0 0) 0x7f8ba4003520 con 0x7f8bb806d9e0 2023-11-24T06:15:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2457923829 >> v2:172.21.15.6:3300/0 conn(0x7f8bb815f270 msgr2=0x7f8bb80cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/2457923829 >> v2:172.21.15.6:3300/0 conn(0x7f8bb815f270 0x7f8bb80cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2457923829 >> v1:172.21.15.6:6789/0 conn(0x7f8bb8144770 legacy=0x7f8bb815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2457923829 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb80b3df0 con 0x7f8bb806d9e0 2023-11-24T06:15:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2457923829 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f8bb00019f0 con 0x7f8bb806d9e0 2023-11-24T06:15:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2457923829 >> v1:172.21.15.5:6789/0 conn(0x7f8bb806d9e0 legacy=0x7f8bb80cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/2457923829 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ba4003a30 0x7f8ba4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/2457923829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ba40047c0 0x7f8ba4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8bb6ffd640 1 --2- 172.21.15.5:0/2457923829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ba40051d0 0x7f8ba400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2457923829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80b43b0 con 0x7f8ba40047c0 2023-11-24T06:15:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2457923829 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba400c100 con 0x7f8ba40051d0 2023-11-24T06:15:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2457923829 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba400c2c0 con 0x7f8ba4003a30 2023-11-24T06:15:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8baffff640 1 --2- 172.21.15.5:0/2457923829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ba40047c0 0x7f8ba4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.282+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/2457923829 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ba4003a30 0x7f8ba4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.283+0000 7f8bb7fff640 1 --2- 172.21.15.5:0/2457923829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ba40051d0 0x7f8ba400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.283+0000 7f8baffff640 1 -- 172.21.15.5:0/2457923829 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ba4003a30 msgr2=0x7f8ba4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.283+0000 7f8baffff640 1 --2- 172.21.15.5:0/2457923829 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8ba4003a30 0x7f8ba4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.283+0000 7f8baffff640 1 -- 172.21.15.5:0/2457923829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ba40051d0 msgr2=0x7f8ba400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.283+0000 7f8baffff640 1 --2- 172.21.15.5:0/2457923829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ba40051d0 0x7f8ba400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.283+0000 7f8baffff640 1 -- 172.21.15.5:0/2457923829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba4003520 con 0x7f8ba40047c0 2023-11-24T06:15:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.283+0000 7f8bb7fff640 1 --2- 172.21.15.5:0/2457923829 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ba40051d0 0x7f8ba400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:15:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.283+0000 7f8bb7fff640 1 -- 172.21.15.5:0/2457923829 reap_dead start 2023-11-24T06:15:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.284+0000 7f8baffff640 1 --2- 172.21.15.5:0/2457923829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ba40047c0 0x7f8ba4004c40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009ac0 tx=0x7f8ba40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8eebfa301170bf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.284+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2457923829 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba002c6b0 con 0x7f8ba40047c0 2023-11-24T06:15:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.284+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2457923829 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba002c810 con 0x7f8ba40047c0 2023-11-24T06:15:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.284+0000 7f8bb6ffd640 1 -- 172.21.15.5:0/2457923829 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba002cba0 con 0x7f8ba40047c0 2023-11-24T06:15:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.285+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/2457923829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ba40047c0 msgr2=0x7f8ba4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.285+0000 7f8bbcf9a640 1 --2- 172.21.15.5:0/2457923829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ba40047c0 0x7f8ba4004c40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009ac0 tx=0x7f8ba40057f0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.285+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/2457923829 shutdown_connections 2023-11-24T06:15:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.285+0000 7f8bbcf9a640 1 --2- 172.21.15.5:0/2457923829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ba40047c0 0x7f8ba4004c40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.285+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/2457923829 >> 172.21.15.5:0/2457923829 conn(0x7f8bb80cec10 msgr2=0x7f8bb80cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:47.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.285+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/2457923829 shutdown_connections 2023-11-24T06:15:47.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.285+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/2457923829 wait complete. 2023-11-24T06:15:47.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.287+0000 7f8bbcf9a640 1 Processor -- start 2023-11-24T06:15:47.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.287+0000 7f8bbcf9a640 1 -- start start 2023-11-24T06:15:47.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.288+0000 7f8bbcf9a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ba40047c0 0x7f8bb81edf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:47.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.288+0000 7f8bbcf9a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bb80ad820 0x7f8bb80adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:47.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.288+0000 7f8bbcf9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb80a8e70 0x7f8bb80a92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:47.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.288+0000 7f8bbcf9a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80ae1e0 con 0x7f8bb80a8e70 2023-11-24T06:15:47.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.288+0000 7f8bbcf9a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80aa540 con 0x7f8ba40047c0 2023-11-24T06:15:47.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.288+0000 7f8bbcf9a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80a9da0 con 0x7f8bb80ad820 2023-11-24T06:15:47.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.289+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb80a8e70 0x7f8bb80a92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:47.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.289+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb80a8e70 0x7f8bb80a92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57432/0 (socket says 172.21.15.5:57432) 2023-11-24T06:15:47.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.289+0000 7f8bb7fff640 1 -- 172.21.15.5:0/352016044 learned_addr learned my addr 172.21.15.5:0/352016044 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:47.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.289+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/352016044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ba40047c0 0x7f8bb81edf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:47.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.289+0000 7f8baffff640 1 --2- 172.21.15.5:0/352016044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bb80ad820 0x7f8bb80adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:47.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.289+0000 7f8bb7fff640 1 -- 172.21.15.5:0/352016044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bb80ad820 msgr2=0x7f8bb80adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.289+0000 7f8bb7fff640 1 --2- 172.21.15.5:0/352016044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bb80ad820 0x7f8bb80adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.289+0000 7f8bb7fff640 1 -- 172.21.15.5:0/352016044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ba40047c0 msgr2=0x7f8bb81edf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.289+0000 7f8bb7fff640 1 --2- 172.21.15.5:0/352016044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ba40047c0 0x7f8bb81edf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.289+0000 7f8bb7fff640 1 -- 172.21.15.5:0/352016044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb814bcd0 con 0x7f8bb80a8e70 2023-11-24T06:15:47.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.289+0000 7f8bb7fff640 1 --2- 172.21.15.5:0/352016044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb80a8e70 0x7f8bb80a92f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80ccd00 tx=0x7f8bb000df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:47.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.290+0000 7f8bb4ff9640 1 -- 172.21.15.5:0/352016044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000b830 con 0x7f8bb80a8e70 2023-11-24T06:15:47.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.290+0000 7f8bb4ff9640 1 -- 172.21.15.5:0/352016044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb000b990 con 0x7f8bb80a8e70 2023-11-24T06:15:47.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.290+0000 7f8bb4ff9640 1 -- 172.21.15.5:0/352016044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000bd00 con 0x7f8bb80a8e70 2023-11-24T06:15:47.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.290+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/352016044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb80a9830 con 0x7f8bb80a8e70 2023-11-24T06:15:47.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.290+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/352016044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb814b680 con 0x7f8bb80a8e70 2023-11-24T06:15:47.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.291+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/352016044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb814be60 con 0x7f8bb80a8e70 2023-11-24T06:15:47.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.292+0000 7f8bb4ff9640 1 -- 172.21.15.5:0/352016044 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8bb0007070 con 0x7f8bb80a8e70 2023-11-24T06:15:47.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.293+0000 7f8bb4ff9640 1 --2- 172.21.15.5:0/352016044 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8b9806ae70 0x7f8b9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:47.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.293+0000 7f8bb4ff9640 1 -- 172.21.15.5:0/352016044 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8bb0088f90 con 0x7f8bb80a8e70 2023-11-24T06:15:47.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.293+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/352016044 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8b9806ae70 0x7f8b9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:47.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.294+0000 7f8bb77fe640 1 --2- 172.21.15.5:0/352016044 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8b9806ae70 0x7f8b9806d330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8007fd0 tx=0x7f8ba8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:47.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.297+0000 7f8bb4ff9640 1 -- 172.21.15.5:0/352016044 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8bb0054380 con 0x7f8bb80a8e70 2023-11-24T06:15:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.492+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/352016044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8bb804e840 con 0x7f8bb80a8e70 2023-11-24T06:15:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.493+0000 7f8bb4ff9640 1 -- 172.21.15.5:0/352016044 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8bb00579a0 con 0x7f8bb80a8e70 2023-11-24T06:15:47.496 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:15:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.496+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/352016044 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8b9806ae70 msgr2=0x7f8b9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.496+0000 7f8bbcf9a640 1 --2- 172.21.15.5:0/352016044 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8b9806ae70 0x7f8b9806d330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8007fd0 tx=0x7f8ba8009040 comp rx=0 tx=0).stop 2023-11-24T06:15:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.496+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/352016044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb80a8e70 msgr2=0x7f8bb80a92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.496+0000 7f8bbcf9a640 1 --2- 172.21.15.5:0/352016044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb80a8e70 0x7f8bb80a92f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80ccd00 tx=0x7f8bb000df50 comp rx=0 tx=0).stop 2023-11-24T06:15:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.496+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/352016044 shutdown_connections 2023-11-24T06:15:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.496+0000 7f8bbcf9a640 1 --2- 172.21.15.5:0/352016044 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8b9806ae70 0x7f8b9806d330 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.496+0000 7f8bbcf9a640 1 --2- 172.21.15.5:0/352016044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8bb80ad820 0x7f8bb80adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.496+0000 7f8bbcf9a640 1 --2- 172.21.15.5:0/352016044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bb80a8e70 0x7f8bb80a92f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.496+0000 7f8bbcf9a640 1 --2- 172.21.15.5:0/352016044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8ba40047c0 0x7f8bb81edf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.496+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/352016044 >> 172.21.15.5:0/352016044 conn(0x7f8bb80cec10 msgr2=0x7f8bb8070880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.496+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/352016044 shutdown_connections 2023-11-24T06:15:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.496+0000 7f8bbcf9a640 1 -- 172.21.15.5:0/352016044 wait complete. 2023-11-24T06:15:47.519 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:15:47.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.726+0000 7f18d98b4640 1 Processor -- start 2023-11-24T06:15:47.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.726+0000 7f18d98b4640 1 -- start start 2023-11-24T06:15:47.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.728+0000 7f18d98b4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f18d40b39a0 0x7f18d4071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:47.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.728+0000 7f18d98b4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f18d40abff0 0x7f18d4071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:47.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.728+0000 7f18d98b4640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d413da30 con 0x7f18d415a100 2023-11-24T06:15:47.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.728+0000 7f18d98b4640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d40ac840 con 0x7f18d40abff0 2023-11-24T06:15:47.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.728+0000 7f18d98b4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d40ac560 con 0x7f18d40b39a0 2023-11-24T06:15:47.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.728+0000 7f18d2ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f18d415a100 0x7f18d40b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48026/0 (socket says 172.21.15.5:48026) 2023-11-24T06:15:47.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.728+0000 7f18d2ffd640 1 -- 172.21.15.5:0/1206023772 learned_addr learned my addr 172.21.15.5:0/1206023772 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:47.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.728+0000 7f18d37fe640 1 --2- 172.21.15.5:0/1206023772 >> v2:172.21.15.5:3300/0 conn(0x7f18d40abff0 0x7f18d4071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:47.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.729+0000 7f18cbfff640 1 --2- 172.21.15.5:0/1206023772 >> v2:172.21.15.6:3300/0 conn(0x7f18d40b39a0 0x7f18d4071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:47.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.729+0000 7f18d37fe640 1 -- 172.21.15.5:0/1206023772 >> v2:172.21.15.6:3300/0 conn(0x7f18d40b39a0 msgr2=0x7f18d4071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:47.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.729+0000 7f18d37fe640 1 --2- 172.21.15.5:0/1206023772 >> v2:172.21.15.6:3300/0 conn(0x7f18d40b39a0 0x7f18d4071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.729+0000 7f18d37fe640 1 -- 172.21.15.5:0/1206023772 >> v1:172.21.15.5:6789/0 conn(0x7f18d415a100 legacy=0x7f18d40b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.729+0000 7f18d37fe640 1 -- 172.21.15.5:0/1206023772 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d4070650 con 0x7f18d40abff0 2023-11-24T06:15:47.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.729+0000 7f18d27fc640 1 -- 172.21.15.5:0/1206023772 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377806527 0 0) 0x7f18d413da30 con 0x7f18d415a100 2023-11-24T06:15:47.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.730+0000 7f18d37fe640 1 --2- 172.21.15.5:0/1206023772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d40abff0 0x7f18d4071670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f18d406fe60 tx=0x7f18cc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c85b0ed4830232 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:47.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.730+0000 7f18d27fc640 1 -- 172.21.15.5:0/1206023772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc02cc80 con 0x7f18d40abff0 2023-11-24T06:15:47.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.730+0000 7f18d27fc640 1 -- 172.21.15.5:0/1206023772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18cc02cde0 con 0x7f18d40abff0 2023-11-24T06:15:47.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.730+0000 7f18d27fc640 1 -- 172.21.15.5:0/1206023772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc034660 con 0x7f18d40abff0 2023-11-24T06:15:47.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.731+0000 7f18d98b4640 1 -- 172.21.15.5:0/1206023772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d40abff0 msgr2=0x7f18d4071670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.731+0000 7f18d98b4640 1 --2- 172.21.15.5:0/1206023772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d40abff0 0x7f18d4071670 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f18d406fe60 tx=0x7f18cc02bed0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.731+0000 7f18d98b4640 1 -- 172.21.15.5:0/1206023772 shutdown_connections 2023-11-24T06:15:47.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.731+0000 7f18d98b4640 1 --2- 172.21.15.5:0/1206023772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d40abff0 0x7f18d4071670 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.731+0000 7f18d98b4640 1 --2- 172.21.15.5:0/1206023772 >> v2:172.21.15.6:3300/0 conn(0x7f18d40b39a0 0x7f18d4071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.731+0000 7f18d98b4640 1 -- 172.21.15.5:0/1206023772 >> 172.21.15.5:0/1206023772 conn(0x7f18d40b0d80 msgr2=0x7f18d40b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:47.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.731+0000 7f18d98b4640 1 -- 172.21.15.5:0/1206023772 shutdown_connections 2023-11-24T06:15:47.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.731+0000 7f18d98b4640 1 -- 172.21.15.5:0/1206023772 wait complete. 2023-11-24T06:15:47.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.733+0000 7f18d98b4640 1 Processor -- start 2023-11-24T06:15:47.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.733+0000 7f18d98b4640 1 -- start start 2023-11-24T06:15:47.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.734+0000 7f18d98b4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d40abff0 0x7f18d41f0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:47.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.734+0000 7f18d98b4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18d40b39a0 0x7f18d414f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:47.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.734+0000 7f18d98b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d415a100 0x7f18d414fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:47.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.734+0000 7f18d98b4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d41518b0 con 0x7f18d415a100 2023-11-24T06:15:47.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.734+0000 7f18d98b4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d4071ee0 con 0x7f18d40abff0 2023-11-24T06:15:47.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.734+0000 7f18d98b4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d40cdb40 con 0x7f18d40b39a0 2023-11-24T06:15:47.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.735+0000 7f18d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d415a100 0x7f18d414fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:47.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.735+0000 7f18d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d415a100 0x7f18d414fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57446/0 (socket says 172.21.15.5:57446) 2023-11-24T06:15:47.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.735+0000 7f18d37fe640 1 -- 172.21.15.5:0/656525811 learned_addr learned my addr 172.21.15.5:0/656525811 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:47.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.735+0000 7f18cbfff640 1 --2- 172.21.15.5:0/656525811 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18d40b39a0 0x7f18d414f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:47.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.735+0000 7f18d2ffd640 1 --2- 172.21.15.5:0/656525811 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d40abff0 0x7f18d41f0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:47.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.735+0000 7f18d37fe640 1 -- 172.21.15.5:0/656525811 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18d40b39a0 msgr2=0x7f18d414f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.735+0000 7f18d37fe640 1 --2- 172.21.15.5:0/656525811 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18d40b39a0 0x7f18d414f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.735+0000 7f18d37fe640 1 -- 172.21.15.5:0/656525811 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d40abff0 msgr2=0x7f18d41f0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.735+0000 7f18d37fe640 1 --2- 172.21.15.5:0/656525811 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d40abff0 0x7f18d41f0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.735+0000 7f18d37fe640 1 -- 172.21.15.5:0/656525811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc009a70 con 0x7f18d415a100 2023-11-24T06:15:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.735+0000 7f18d37fe640 1 --2- 172.21.15.5:0/656525811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d415a100 0x7f18d414fdb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18cc02c4e0 tx=0x7f18cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.736+0000 7f18d0ff9640 1 -- 172.21.15.5:0/656525811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc040650 con 0x7f18d415a100 2023-11-24T06:15:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.736+0000 7f18d0ff9640 1 -- 172.21.15.5:0/656525811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18cc0407b0 con 0x7f18d415a100 2023-11-24T06:15:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.736+0000 7f18d98b4640 1 -- 172.21.15.5:0/656525811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d4151ae0 con 0x7f18d415a100 2023-11-24T06:15:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.736+0000 7f18d0ff9640 1 -- 172.21.15.5:0/656525811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc040b60 con 0x7f18d415a100 2023-11-24T06:15:47.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.736+0000 7f18d98b4640 1 -- 172.21.15.5:0/656525811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d41572d0 con 0x7f18d415a100 2023-11-24T06:15:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.737+0000 7f18d98b4640 1 -- 172.21.15.5:0/656525811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d4067960 con 0x7f18d415a100 2023-11-24T06:15:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.738+0000 7f18d0ff9640 1 -- 172.21.15.5:0/656525811 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f18cc040cc0 con 0x7f18d415a100 2023-11-24T06:15:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.739+0000 7f18d0ff9640 1 --2- 172.21.15.5:0/656525811 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f18b406ae40 0x7f18b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:47.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.739+0000 7f18d0ff9640 1 -- 172.21.15.5:0/656525811 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f18cc0ac380 con 0x7f18d415a100 2023-11-24T06:15:47.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.739+0000 7f18d2ffd640 1 --2- 172.21.15.5:0/656525811 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f18b406ae40 0x7f18b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:47.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.740+0000 7f18d2ffd640 1 --2- 172.21.15.5:0/656525811 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f18b406ae40 0x7f18b406d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f18c4007fd0 tx=0x7f18c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:47.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.743+0000 7f18d0ff9640 1 -- 172.21.15.5:0/656525811 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f18cc077770 con 0x7f18d415a100 2023-11-24T06:15:47.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.979+0000 7f18d98b4640 1 -- 172.21.15.5:0/656525811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18d40cdff0 con 0x7f18d415a100 2023-11-24T06:15:47.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.980+0000 7f18d0ff9640 1 -- 172.21.15.5:0/656525811 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3682 (secure 0 0 0) 0x7f18d40cdff0 con 0x7f18d415a100 2023-11-24T06:15:47.983 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:15:47.983 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4009460946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4009460946},{"type":"v1","addr":"172.21.15.5:6837","nonce":4009460946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2496926427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2496926427},{"type":"v1","addr":"172.21.15.5:6836","nonce":2496926427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3829754769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3829754769},{"type":"v1","addr":"172.21.15.6:6835","nonce":3829754769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6833/2760317954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2760317954},{"type":"v1","addr":"172.21.15.6:6833","nonce":2760317954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-24T06:15:47.983 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2023-11-24T06:15:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.983+0000 7f18d98b4640 1 -- 172.21.15.5:0/656525811 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f18b406ae40 msgr2=0x7f18b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.983+0000 7f18d98b4640 1 --2- 172.21.15.5:0/656525811 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f18b406ae40 0x7f18b406d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f18c4007fd0 tx=0x7f18c4009040 comp rx=0 tx=0).stop 2023-11-24T06:15:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.983+0000 7f18d98b4640 1 -- 172.21.15.5:0/656525811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d415a100 msgr2=0x7f18d414fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.983+0000 7f18d98b4640 1 --2- 172.21.15.5:0/656525811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d415a100 0x7f18d414fdb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f18cc02c4e0 tx=0x7f18cc004a40 comp rx=0 tx=0).stop 2023-11-24T06:15:47.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.983+0000 7f18d98b4640 1 -- 172.21.15.5:0/656525811 shutdown_connections 2023-11-24T06:15:47.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.983+0000 7f18d98b4640 1 --2- 172.21.15.5:0/656525811 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f18b406ae40 0x7f18b406d300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.983+0000 7f18d98b4640 1 --2- 172.21.15.5:0/656525811 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f18d40b39a0 0x7f18d414f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.983+0000 7f18d98b4640 1 --2- 172.21.15.5:0/656525811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d415a100 0x7f18d414fdb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.983+0000 7f18d98b4640 1 --2- 172.21.15.5:0/656525811 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f18d40abff0 0x7f18d41f0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:47.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.983+0000 7f18d98b4640 1 -- 172.21.15.5:0/656525811 >> 172.21.15.5:0/656525811 conn(0x7f18d40b0d80 msgr2=0x7f18d40b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:47.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.983+0000 7f18d98b4640 1 -- 172.21.15.5:0/656525811 shutdown_connections 2023-11-24T06:15:47.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:47.983+0000 7f18d98b4640 1 -- 172.21.15.5:0/656525811 wait complete. 2023-11-24T06:15:48.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:15:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.220+0000 7f633902e640 1 Processor -- start 2023-11-24T06:15:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.221+0000 7f633902e640 1 -- start start 2023-11-24T06:15:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.223+0000 7f633902e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f63340ce2a0 0x7f6334167f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.223+0000 7f633902e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63340cbf10 con 0x7f63340cd6e0 2023-11-24T06:15:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.223+0000 7f633902e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63340cb5e0 con 0x7f633415d550 2023-11-24T06:15:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.223+0000 7f633902e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63340d05e0 con 0x7f63340ce2a0 2023-11-24T06:15:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.223+0000 7f63337fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f63340cd6e0 0x7f63340cdb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48042/0 (socket says 172.21.15.5:48042) 2023-11-24T06:15:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.223+0000 7f63337fe640 1 -- 172.21.15.5:0/3390871264 learned_addr learned my addr 172.21.15.5:0/3390871264 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.223+0000 7f633882d640 1 --2- 172.21.15.5:0/3390871264 >> v2:172.21.15.6:3300/0 conn(0x7f63340ce2a0 0x7f6334167f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.223+0000 7f6332ffd640 1 -- 172.21.15.5:0/3390871264 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375469436 0 0) 0x7f63340cbf10 con 0x7f63340cd6e0 2023-11-24T06:15:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f6332ffd640 1 -- 172.21.15.5:0/3390871264 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f631c003520 con 0x7f63340cd6e0 2023-11-24T06:15:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f6332ffd640 1 -- 172.21.15.5:0/3390871264 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1854613141 0 0) 0x7f631c003520 con 0x7f63340cd6e0 2023-11-24T06:15:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f6332ffd640 1 -- 172.21.15.5:0/3390871264 >> v2:172.21.15.6:3300/0 conn(0x7f63340ce2a0 msgr2=0x7f6334167f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f6332ffd640 1 --2- 172.21.15.5:0/3390871264 >> v2:172.21.15.6:3300/0 conn(0x7f63340ce2a0 0x7f6334167f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f6332ffd640 1 -- 172.21.15.5:0/3390871264 >> v1:172.21.15.6:6789/0 conn(0x7f633415d550 legacy=0x7f63340cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f6332ffd640 1 -- 172.21.15.5:0/3390871264 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63340cb8c0 con 0x7f63340cd6e0 2023-11-24T06:15:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f6332ffd640 1 -- 172.21.15.5:0/3390871264 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f6328001670 con 0x7f63340cd6e0 2023-11-24T06:15:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f6332ffd640 1 -- 172.21.15.5:0/3390871264 >> v1:172.21.15.5:6789/0 conn(0x7f63340cd6e0 legacy=0x7f63340cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f6332ffd640 1 --2- 172.21.15.5:0/3390871264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f631c0039e0 0x7f631c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f6332ffd640 1 --2- 172.21.15.5:0/3390871264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f631c0047c0 0x7f631c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f6332ffd640 1 --2- 172.21.15.5:0/3390871264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f631c0051d0 0x7f631c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f6332ffd640 1 -- 172.21.15.5:0/3390871264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63340d05e0 con 0x7f631c0051d0 2023-11-24T06:15:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f6332ffd640 1 -- 172.21.15.5:0/3390871264 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c00c100 con 0x7f631c0047c0 2023-11-24T06:15:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f6332ffd640 1 -- 172.21.15.5:0/3390871264 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631c00c2c0 con 0x7f631c0039e0 2023-11-24T06:15:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.224+0000 7f633882d640 1 --2- 172.21.15.5:0/3390871264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f631c0051d0 0x7f631c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.225+0000 7f6333fff640 1 --2- 172.21.15.5:0/3390871264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f631c0039e0 0x7f631c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.225+0000 7f63337fe640 1 --2- 172.21.15.5:0/3390871264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f631c0047c0 0x7f631c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.225+0000 7f633882d640 1 -- 172.21.15.5:0/3390871264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f631c0039e0 msgr2=0x7f631c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.225+0000 7f633882d640 1 --2- 172.21.15.5:0/3390871264 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f631c0039e0 0x7f631c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.225+0000 7f633882d640 1 -- 172.21.15.5:0/3390871264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f631c0047c0 msgr2=0x7f631c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.225+0000 7f633882d640 1 --2- 172.21.15.5:0/3390871264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f631c0047c0 0x7f631c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.225+0000 7f633882d640 1 -- 172.21.15.5:0/3390871264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c003520 con 0x7f631c0051d0 2023-11-24T06:15:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.225+0000 7f63337fe640 1 --2- 172.21.15.5:0/3390871264 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f631c0047c0 0x7f631c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:15:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.226+0000 7f633882d640 1 -- 172.21.15.5:0/3390871264 reap_dead start 2023-11-24T06:15:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.226+0000 7f633882d640 1 --2- 172.21.15.5:0/3390871264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f631c0051d0 0x7f631c00b980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f632c009fc0 tx=0x7f631c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e288baa9731516e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.226+0000 7f6332ffd640 1 -- 172.21.15.5:0/3390871264 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c02cc60 con 0x7f631c0051d0 2023-11-24T06:15:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.226+0000 7f6332ffd640 1 -- 172.21.15.5:0/3390871264 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632c00f040 con 0x7f631c0051d0 2023-11-24T06:15:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.226+0000 7f6332ffd640 1 -- 172.21.15.5:0/3390871264 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c003990 con 0x7f631c0051d0 2023-11-24T06:15:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.227+0000 7f633902e640 1 -- 172.21.15.5:0/3390871264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f631c0051d0 msgr2=0x7f631c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.227+0000 7f633902e640 1 --2- 172.21.15.5:0/3390871264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f631c0051d0 0x7f631c00b980 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f632c009fc0 tx=0x7f631c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.227+0000 7f633902e640 1 -- 172.21.15.5:0/3390871264 shutdown_connections 2023-11-24T06:15:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.227+0000 7f633902e640 1 --2- 172.21.15.5:0/3390871264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f631c0051d0 0x7f631c00b980 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.227+0000 7f633902e640 1 -- 172.21.15.5:0/3390871264 >> 172.21.15.5:0/3390871264 conn(0x7f63340d0e60 msgr2=0x7f63340d1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.227+0000 7f633902e640 1 -- 172.21.15.5:0/3390871264 shutdown_connections 2023-11-24T06:15:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.227+0000 7f633902e640 1 -- 172.21.15.5:0/3390871264 wait complete. 2023-11-24T06:15:48.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.229+0000 7f633902e640 1 Processor -- start 2023-11-24T06:15:48.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.229+0000 7f633902e640 1 -- start start 2023-11-24T06:15:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f633902e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f631c0051d0 0x7f63340c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f633902e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f633415a170 0x7f63340aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f633902e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63341595d0 0x7f6334159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f633902e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63340cada0 con 0x7f631c0051d0 2023-11-24T06:15:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f633902e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6334159f90 con 0x7f633415a170 2023-11-24T06:15:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f633902e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63340c90e0 con 0x7f63341595d0 2023-11-24T06:15:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f6333fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f631c0051d0 0x7f63340c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f6333fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f631c0051d0 0x7f63340c8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57462/0 (socket says 172.21.15.5:57462) 2023-11-24T06:15:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f6333fff640 1 -- 172.21.15.5:0/1861684543 learned_addr learned my addr 172.21.15.5:0/1861684543 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f6333fff640 1 -- 172.21.15.5:0/1861684543 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63341595d0 msgr2=0x7f6334159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f63337fe640 1 --2- 172.21.15.5:0/1861684543 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f633415a170 0x7f63340aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f633882d640 1 --2- 172.21.15.5:0/1861684543 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63341595d0 0x7f6334159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f6333fff640 1 --2- 172.21.15.5:0/1861684543 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63341595d0 0x7f6334159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f6333fff640 1 -- 172.21.15.5:0/1861684543 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f633415a170 msgr2=0x7f63340aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f6333fff640 1 --2- 172.21.15.5:0/1861684543 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f633415a170 0x7f63340aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.230+0000 7f6333fff640 1 -- 172.21.15.5:0/1861684543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63340c8e60 con 0x7f631c0051d0 2023-11-24T06:15:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.231+0000 7f6333fff640 1 --2- 172.21.15.5:0/1861684543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f631c0051d0 0x7f63340c8920 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f632400e5f0 tx=0x7f632400eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.231+0000 7f6330ff9640 1 -- 172.21.15.5:0/1861684543 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63240052e0 con 0x7f631c0051d0 2023-11-24T06:15:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.231+0000 7f6330ff9640 1 -- 172.21.15.5:0/1861684543 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6324005440 con 0x7f631c0051d0 2023-11-24T06:15:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.231+0000 7f6330ff9640 1 -- 172.21.15.5:0/1861684543 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324003830 con 0x7f631c0051d0 2023-11-24T06:15:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.231+0000 7f633902e640 1 -- 172.21.15.5:0/1861684543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63340ca540 con 0x7f631c0051d0 2023-11-24T06:15:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.231+0000 7f633902e640 1 -- 172.21.15.5:0/1861684543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63340caac0 con 0x7f631c0051d0 2023-11-24T06:15:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.232+0000 7f633902e640 1 -- 172.21.15.5:0/1861684543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633404e840 con 0x7f631c0051d0 2023-11-24T06:15:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.233+0000 7f6330ff9640 1 -- 172.21.15.5:0/1861684543 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6324003990 con 0x7f631c0051d0 2023-11-24T06:15:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.234+0000 7f6330ff9640 1 --2- 172.21.15.5:0/1861684543 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f632006ae40 0x7f632006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.234+0000 7f6330ff9640 1 -- 172.21.15.5:0/1861684543 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6324089110 con 0x7f631c0051d0 2023-11-24T06:15:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.235+0000 7f63337fe640 1 --2- 172.21.15.5:0/1861684543 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f632006ae40 0x7f632006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.235+0000 7f63337fe640 1 --2- 172.21.15.5:0/1861684543 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f632006ae40 0x7f632006d300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f63340afc40 tx=0x7f6328009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:48.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.239+0000 7f6330ff9640 1 -- 172.21.15.5:0/1861684543 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6324054660 con 0x7f631c0051d0 2023-11-24T06:15:48.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.428+0000 7f633902e640 1 -- 172.21.15.5:0/1861684543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f633406f770 con 0x7f631c0051d0 2023-11-24T06:15:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.429+0000 7f6330ff9640 1 -- 172.21.15.5:0/1861684543 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12706 (secure 0 0 0) 0x7f6324057c80 con 0x7f631c0051d0 2023-11-24T06:15:48.433 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:15:48.433 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":18,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:42.775578+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:15:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.431+0000 7f633902e640 1 -- 172.21.15.5:0/1861684543 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f632006ae40 msgr2=0x7f632006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.432+0000 7f633902e640 1 --2- 172.21.15.5:0/1861684543 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f632006ae40 0x7f632006d300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f63340afc40 tx=0x7f6328009040 comp rx=0 tx=0).stop 2023-11-24T06:15:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.432+0000 7f633902e640 1 -- 172.21.15.5:0/1861684543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f631c0051d0 msgr2=0x7f63340c8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.432+0000 7f633902e640 1 --2- 172.21.15.5:0/1861684543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f631c0051d0 0x7f63340c8920 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f632400e5f0 tx=0x7f632400eab0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.432+0000 7f633902e640 1 -- 172.21.15.5:0/1861684543 shutdown_connections 2023-11-24T06:15:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.432+0000 7f633902e640 1 --2- 172.21.15.5:0/1861684543 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f632006ae40 0x7f632006d300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.432+0000 7f633902e640 1 --2- 172.21.15.5:0/1861684543 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63341595d0 0x7f6334159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.432+0000 7f633902e640 1 --2- 172.21.15.5:0/1861684543 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f633415a170 0x7f63340aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.432+0000 7f633902e640 1 --2- 172.21.15.5:0/1861684543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f631c0051d0 0x7f63340c8920 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.432+0000 7f633902e640 1 -- 172.21.15.5:0/1861684543 >> 172.21.15.5:0/1861684543 conn(0x7f63340d0e60 msgr2=0x7f633406c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.432+0000 7f633902e640 1 -- 172.21.15.5:0/1861684543 shutdown_connections 2023-11-24T06:15:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.432+0000 7f633902e640 1 -- 172.21.15.5:0/1861684543 wait complete. 2023-11-24T06:15:48.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:15:48.455 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:15:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.675+0000 7f325511d640 1 Processor -- start 2023-11-24T06:15:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.675+0000 7f325511d640 1 -- start start 2023-11-24T06:15:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.677+0000 7f325511d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f325006cad0 0x7f325006e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.677+0000 7f325511d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3250071780 con 0x7f3250072280 2023-11-24T06:15:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.677+0000 7f325511d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32500711c0 con 0x7f3250160b10 2023-11-24T06:15:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.677+0000 7f325511d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32500b2320 con 0x7f325006cad0 2023-11-24T06:15:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.677+0000 7f324e59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f325006cad0 0x7f325006e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.677+0000 7f324f59e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3250072280 0x7f32500b2740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48052/0 (socket says 172.21.15.5:48052) 2023-11-24T06:15:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.677+0000 7f324f59e640 1 -- 172.21.15.5:0/3584529566 learned_addr learned my addr 172.21.15.5:0/3584529566 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.678+0000 7f324e59c640 1 -- 172.21.15.5:0/3584529566 >> v1:172.21.15.6:6789/0 conn(0x7f3250160b10 legacy=0x7f325006c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.678+0000 7f324e59c640 1 -- 172.21.15.5:0/3584529566 >> v1:172.21.15.5:6789/0 conn(0x7f3250072280 legacy=0x7f32500b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.678+0000 7f324dd9b640 1 -- 172.21.15.5:0/3584529566 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676702954 0 0) 0x7f3250071780 con 0x7f3250072280 2023-11-24T06:15:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.678+0000 7f324e59c640 1 -- 172.21.15.5:0/3584529566 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32500714a0 con 0x7f325006cad0 2023-11-24T06:15:48.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.679+0000 7f324e59c640 1 --2- 172.21.15.5:0/3584529566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325006cad0 0x7f325006e420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f32380097a0 tx=0x7f323802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1ffdebea2fdbce server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:48.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.679+0000 7f324dd9b640 1 -- 172.21.15.5:0/3584529566 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238031830 con 0x7f325006cad0 2023-11-24T06:15:48.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.679+0000 7f324dd9b640 1 -- 172.21.15.5:0/3584529566 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3238031990 con 0x7f325006cad0 2023-11-24T06:15:48.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.679+0000 7f324dd9b640 1 -- 172.21.15.5:0/3584529566 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238031d20 con 0x7f325006cad0 2023-11-24T06:15:48.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.680+0000 7f325511d640 1 -- 172.21.15.5:0/3584529566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325006cad0 msgr2=0x7f325006e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:48.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.680+0000 7f325511d640 1 --2- 172.21.15.5:0/3584529566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325006cad0 0x7f325006e420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f32380097a0 tx=0x7f323802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.680+0000 7f325511d640 1 -- 172.21.15.5:0/3584529566 shutdown_connections 2023-11-24T06:15:48.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.680+0000 7f325511d640 1 --2- 172.21.15.5:0/3584529566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325006cad0 0x7f325006e420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.680+0000 7f325511d640 1 -- 172.21.15.5:0/3584529566 >> 172.21.15.5:0/3584529566 conn(0x7f32500b0ba0 msgr2=0x7f32500b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:48.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.680+0000 7f325511d640 1 -- 172.21.15.5:0/3584529566 shutdown_connections 2023-11-24T06:15:48.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.680+0000 7f325511d640 1 -- 172.21.15.5:0/3584529566 wait complete. 2023-11-24T06:15:48.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.682+0000 7f325511d640 1 Processor -- start 2023-11-24T06:15:48.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.682+0000 7f325511d640 1 -- start start 2023-11-24T06:15:48.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.683+0000 7f325511d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325006cad0 0x7f325014dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:48.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.683+0000 7f325511d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3250072280 0x7f32500cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:48.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.683+0000 7f325511d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3250160b10 0x7f32500cf230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:48.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.683+0000 7f325511d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325014e0e0 con 0x7f325006cad0 2023-11-24T06:15:48.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.683+0000 7f325511d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32500cc780 con 0x7f3250072280 2023-11-24T06:15:48.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.683+0000 7f325511d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32500cbb70 con 0x7f3250160b10 2023-11-24T06:15:48.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.683+0000 7f324ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325006cad0 0x7f325014dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:48.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.683+0000 7f324ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325006cad0 0x7f325014dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57472/0 (socket says 172.21.15.5:57472) 2023-11-24T06:15:48.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.683+0000 7f324ed9d640 1 -- 172.21.15.5:0/742632848 learned_addr learned my addr 172.21.15.5:0/742632848 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:48.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.683+0000 7f324e59c640 1 --2- 172.21.15.5:0/742632848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3250072280 0x7f32500cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:48.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.683+0000 7f324ed9d640 1 -- 172.21.15.5:0/742632848 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3250160b10 msgr2=0x7f32500cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:48.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.684+0000 7f324ed9d640 1 --2- 172.21.15.5:0/742632848 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3250160b10 0x7f32500cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.684+0000 7f324ed9d640 1 -- 172.21.15.5:0/742632848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3250072280 msgr2=0x7f32500cecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:48.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.684+0000 7f324ed9d640 1 --2- 172.21.15.5:0/742632848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3250072280 0x7f32500cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.684+0000 7f324ed9d640 1 -- 172.21.15.5:0/742632848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3238009400 con 0x7f325006cad0 2023-11-24T06:15:48.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.684+0000 7f324ed9d640 1 --2- 172.21.15.5:0/742632848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325006cad0 0x7f325014dba0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f324000c400 tx=0x7f324000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:48.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.684+0000 7f32477fe640 1 -- 172.21.15.5:0/742632848 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32400052d0 con 0x7f325006cad0 2023-11-24T06:15:48.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.684+0000 7f32477fe640 1 -- 172.21.15.5:0/742632848 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3240005430 con 0x7f325006cad0 2023-11-24T06:15:48.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.684+0000 7f325511d640 1 -- 172.21.15.5:0/742632848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32500cbe50 con 0x7f325006cad0 2023-11-24T06:15:48.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.684+0000 7f32477fe640 1 -- 172.21.15.5:0/742632848 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240003830 con 0x7f325006cad0 2023-11-24T06:15:48.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.684+0000 7f325511d640 1 -- 172.21.15.5:0/742632848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32500cc410 con 0x7f325006cad0 2023-11-24T06:15:48.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.685+0000 7f325511d640 1 -- 172.21.15.5:0/742632848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32500acb40 con 0x7f325006cad0 2023-11-24T06:15:48.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.686+0000 7f32477fe640 1 -- 172.21.15.5:0/742632848 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3240003990 con 0x7f325006cad0 2023-11-24T06:15:48.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.687+0000 7f32477fe640 1 --2- 172.21.15.5:0/742632848 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f323006ae40 0x7f323006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:48.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.687+0000 7f32477fe640 1 -- 172.21.15.5:0/742632848 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3240088850 con 0x7f325006cad0 2023-11-24T06:15:48.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.687+0000 7f324e59c640 1 --2- 172.21.15.5:0/742632848 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f323006ae40 0x7f323006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:48.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.688+0000 7f324e59c640 1 --2- 172.21.15.5:0/742632848 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f323006ae40 0x7f323006d300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3238036000 tx=0x7f3238008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:48.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.691+0000 7f32477fe640 1 -- 172.21.15.5:0/742632848 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3240053c40 con 0x7f325006cad0 2023-11-24T06:15:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.890+0000 7f325511d640 1 -- 172.21.15.5:0/742632848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f325006df00 con 0x7f325006cad0 2023-11-24T06:15:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.891+0000 7f32477fe640 1 -- 172.21.15.5:0/742632848 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3240057260 con 0x7f325006cad0 2023-11-24T06:15:48.894 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:15:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.894+0000 7f325511d640 1 -- 172.21.15.5:0/742632848 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f323006ae40 msgr2=0x7f323006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.894+0000 7f325511d640 1 --2- 172.21.15.5:0/742632848 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f323006ae40 0x7f323006d300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3238036000 tx=0x7f3238008030 comp rx=0 tx=0).stop 2023-11-24T06:15:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.894+0000 7f325511d640 1 -- 172.21.15.5:0/742632848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325006cad0 msgr2=0x7f325014dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.894+0000 7f325511d640 1 --2- 172.21.15.5:0/742632848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325006cad0 0x7f325014dba0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f324000c400 tx=0x7f324000c8c0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.894+0000 7f325511d640 1 -- 172.21.15.5:0/742632848 shutdown_connections 2023-11-24T06:15:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.894+0000 7f325511d640 1 --2- 172.21.15.5:0/742632848 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f323006ae40 0x7f323006d300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.894+0000 7f325511d640 1 --2- 172.21.15.5:0/742632848 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3250160b10 0x7f32500cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.894+0000 7f325511d640 1 --2- 172.21.15.5:0/742632848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3250072280 0x7f32500cecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.894+0000 7f325511d640 1 --2- 172.21.15.5:0/742632848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325006cad0 0x7f325014dba0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.894+0000 7f325511d640 1 -- 172.21.15.5:0/742632848 >> 172.21.15.5:0/742632848 conn(0x7f32500b0ba0 msgr2=0x7f32500ac240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.894+0000 7f325511d640 1 -- 172.21.15.5:0/742632848 shutdown_connections 2023-11-24T06:15:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:48.894+0000 7f325511d640 1 -- 172.21.15.5:0/742632848 wait complete. 2023-11-24T06:15:48.917 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:15:49.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.135+0000 7f3aff316640 1 Processor -- start 2023-11-24T06:15:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.136+0000 7f3aff316640 1 -- start start 2023-11-24T06:15:49.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.137+0000 7f3aff316640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3af815df80 0x7f3af81680d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.137+0000 7f3aff316640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af8161670 con 0x7f3af8157770 2023-11-24T06:15:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.137+0000 7f3aff316640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af8071730 con 0x7f3af806d250 2023-11-24T06:15:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.137+0000 7f3aff316640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af815e410 con 0x7f3af815df80 2023-11-24T06:15:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.138+0000 7f3afe314640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3af8157770 0x7f3af806cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48068/0 (socket says 172.21.15.5:48068) 2023-11-24T06:15:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.138+0000 7f3afe314640 1 -- 172.21.15.5:0/3331333932 learned_addr learned my addr 172.21.15.5:0/3331333932 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.138+0000 7f3afeb15640 1 --2- 172.21.15.5:0/3331333932 >> v2:172.21.15.6:3300/0 conn(0x7f3af815df80 0x7f3af81680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.138+0000 7f3afd312640 1 -- 172.21.15.5:0/3331333932 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772814 0 0) 0x7f3af8161670 con 0x7f3af8157770 2023-11-24T06:15:49.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.138+0000 7f3afd312640 1 -- 172.21.15.5:0/3331333932 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ae4003520 con 0x7f3af8157770 2023-11-24T06:15:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.138+0000 7f3afd312640 1 -- 172.21.15.5:0/3331333932 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3752673667 0 0) 0x7f3ae4003520 con 0x7f3af8157770 2023-11-24T06:15:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.138+0000 7f3afd312640 1 -- 172.21.15.5:0/3331333932 >> v2:172.21.15.6:3300/0 conn(0x7f3af815df80 msgr2=0x7f3af81680d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.138+0000 7f3afd312640 1 --2- 172.21.15.5:0/3331333932 >> v2:172.21.15.6:3300/0 conn(0x7f3af815df80 0x7f3af81680d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.138+0000 7f3afd312640 1 -- 172.21.15.5:0/3331333932 >> v1:172.21.15.6:6789/0 conn(0x7f3af806d250 legacy=0x7f3af815d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.138+0000 7f3afd312640 1 -- 172.21.15.5:0/3331333932 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af815e610 con 0x7f3af8157770 2023-11-24T06:15:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.138+0000 7f3afd312640 1 -- 172.21.15.5:0/3331333932 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f3af4001670 con 0x7f3af8157770 2023-11-24T06:15:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.138+0000 7f3afd312640 1 -- 172.21.15.5:0/3331333932 >> v1:172.21.15.5:6789/0 conn(0x7f3af8157770 legacy=0x7f3af806cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:49.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.138+0000 7f3afd312640 1 --2- 172.21.15.5:0/3331333932 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ae40039e0 0x7f3ae4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.139+0000 7f3afd312640 1 --2- 172.21.15.5:0/3331333932 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ae40047c0 0x7f3ae4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.139+0000 7f3afd312640 1 --2- 172.21.15.5:0/3331333932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae4005180 0x7f3ae400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.139+0000 7f3afd312640 1 -- 172.21.15.5:0/3331333932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af815e410 con 0x7f3ae4005180 2023-11-24T06:15:49.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.139+0000 7f3afd312640 1 -- 172.21.15.5:0/3331333932 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae400c0b0 con 0x7f3ae40047c0 2023-11-24T06:15:49.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.139+0000 7f3afd312640 1 -- 172.21.15.5:0/3331333932 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae400c270 con 0x7f3ae40039e0 2023-11-24T06:15:49.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.139+0000 7f3afeb15640 1 --2- 172.21.15.5:0/3331333932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae4005180 0x7f3ae400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.139+0000 7f3afe314640 1 --2- 172.21.15.5:0/3331333932 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ae40039e0 0x7f3ae4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.139+0000 7f3afdb13640 1 --2- 172.21.15.5:0/3331333932 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ae40047c0 0x7f3ae4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.139+0000 7f3afeb15640 1 -- 172.21.15.5:0/3331333932 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ae40039e0 msgr2=0x7f3ae4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.139+0000 7f3afeb15640 1 --2- 172.21.15.5:0/3331333932 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ae40039e0 0x7f3ae4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.139+0000 7f3afeb15640 1 -- 172.21.15.5:0/3331333932 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ae40047c0 msgr2=0x7f3ae4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.139+0000 7f3afeb15640 1 --2- 172.21.15.5:0/3331333932 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3ae40047c0 0x7f3ae4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.139+0000 7f3afeb15640 1 -- 172.21.15.5:0/3331333932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae4003520 con 0x7f3ae4005180 2023-11-24T06:15:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.140+0000 7f3afeb15640 1 -- 172.21.15.5:0/3331333932 reap_dead start 2023-11-24T06:15:49.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.140+0000 7f3afeb15640 1 --2- 172.21.15.5:0/3331333932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae4005180 0x7f3ae400b980 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009fc0 tx=0x7f3ae40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55bc0a9201f9731 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:49.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.141+0000 7f3afd312640 1 -- 172.21.15.5:0/3331333932 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af002cc60 con 0x7f3ae4005180 2023-11-24T06:15:49.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.141+0000 7f3afd312640 1 -- 172.21.15.5:0/3331333932 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af000f040 con 0x7f3ae4005180 2023-11-24T06:15:49.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.141+0000 7f3afd312640 1 -- 172.21.15.5:0/3331333932 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af0003990 con 0x7f3ae4005180 2023-11-24T06:15:49.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.141+0000 7f3aff316640 1 -- 172.21.15.5:0/3331333932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae4005180 msgr2=0x7f3ae400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:49.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.141+0000 7f3aff316640 1 --2- 172.21.15.5:0/3331333932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae4005180 0x7f3ae400b980 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009fc0 tx=0x7f3ae40057f0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.141+0000 7f3aff316640 1 -- 172.21.15.5:0/3331333932 shutdown_connections 2023-11-24T06:15:49.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.141+0000 7f3aff316640 1 --2- 172.21.15.5:0/3331333932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ae4005180 0x7f3ae400b980 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.141+0000 7f3aff316640 1 -- 172.21.15.5:0/3331333932 >> 172.21.15.5:0/3331333932 conn(0x7f3af806a860 msgr2=0x7f3af806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:49.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.141+0000 7f3aff316640 1 -- 172.21.15.5:0/3331333932 shutdown_connections 2023-11-24T06:15:49.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.141+0000 7f3aff316640 1 -- 172.21.15.5:0/3331333932 wait complete. 2023-11-24T06:15:49.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.143+0000 7f3aff316640 1 Processor -- start 2023-11-24T06:15:49.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.143+0000 7f3aff316640 1 -- start start 2023-11-24T06:15:49.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.144+0000 7f3aff316640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ae4005180 0x7f3af814c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.144+0000 7f3aff316640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3af8154ba0 0x7f3af80b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.144+0000 7f3aff316640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3af814b2f0 0x7f3af814b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.144+0000 7f3aff316640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af814bcb0 con 0x7f3af814b2f0 2023-11-24T06:15:49.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.144+0000 7f3aff316640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af8150f80 con 0x7f3af8154ba0 2023-11-24T06:15:49.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.144+0000 7f3aff316640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af81511d0 con 0x7f3ae4005180 2023-11-24T06:15:49.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.145+0000 7f3afeb15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3af814b2f0 0x7f3af814b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.145+0000 7f3afeb15640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3af814b2f0 0x7f3af814b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57496/0 (socket says 172.21.15.5:57496) 2023-11-24T06:15:49.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.145+0000 7f3afeb15640 1 -- 172.21.15.5:0/1490411804 learned_addr learned my addr 172.21.15.5:0/1490411804 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:49.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.145+0000 7f3afdb13640 1 --2- 172.21.15.5:0/1490411804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3af8154ba0 0x7f3af80b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.145+0000 7f3afe314640 1 --2- 172.21.15.5:0/1490411804 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ae4005180 0x7f3af814c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.145+0000 7f3afeb15640 1 -- 172.21.15.5:0/1490411804 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ae4005180 msgr2=0x7f3af814c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:49.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.145+0000 7f3afeb15640 1 --2- 172.21.15.5:0/1490411804 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ae4005180 0x7f3af814c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.145+0000 7f3afeb15640 1 -- 172.21.15.5:0/1490411804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3af8154ba0 msgr2=0x7f3af80b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:49.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.145+0000 7f3afeb15640 1 --2- 172.21.15.5:0/1490411804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3af8154ba0 0x7f3af80b3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.145+0000 7f3afeb15640 1 -- 172.21.15.5:0/1490411804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af81514b0 con 0x7f3af814b2f0 2023-11-24T06:15:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.145+0000 7f3afeb15640 1 --2- 172.21.15.5:0/1490411804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3af814b2f0 0x7f3af814b770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3af000e200 tx=0x7f3af0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.145+0000 7f3aeeffd640 1 -- 172.21.15.5:0/1490411804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af0003da0 con 0x7f3af814b2f0 2023-11-24T06:15:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.146+0000 7f3aeeffd640 1 -- 172.21.15.5:0/1490411804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3af0035980 con 0x7f3af814b2f0 2023-11-24T06:15:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.146+0000 7f3aeeffd640 1 -- 172.21.15.5:0/1490411804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3af0035d30 con 0x7f3af814b2f0 2023-11-24T06:15:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.146+0000 7f3aff316640 1 -- 172.21.15.5:0/1490411804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af8151790 con 0x7f3af814b2f0 2023-11-24T06:15:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.146+0000 7f3aff316640 1 -- 172.21.15.5:0/1490411804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af8151d50 con 0x7f3af814b2f0 2023-11-24T06:15:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.147+0000 7f3aff316640 1 -- 172.21.15.5:0/1490411804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af8151ee0 con 0x7f3af814b2f0 2023-11-24T06:15:49.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.148+0000 7f3aeeffd640 1 -- 172.21.15.5:0/1490411804 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3af0004030 con 0x7f3af814b2f0 2023-11-24T06:15:49.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.149+0000 7f3aeeffd640 1 --2- 172.21.15.5:0/1490411804 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3adc06ae40 0x7f3adc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.149+0000 7f3aeeffd640 1 -- 172.21.15.5:0/1490411804 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3af00abff0 con 0x7f3af814b2f0 2023-11-24T06:15:49.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.149+0000 7f3afe314640 1 --2- 172.21.15.5:0/1490411804 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3adc06ae40 0x7f3adc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.150+0000 7f3afe314640 1 --2- 172.21.15.5:0/1490411804 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3adc06ae40 0x7f3adc06d300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3af4001eb0 tx=0x7f3af4000ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:49.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.153+0000 7f3aeeffd640 1 -- 172.21.15.5:0/1490411804 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3af00774b0 con 0x7f3af814b2f0 2023-11-24T06:15:49.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.339+0000 7f3aff316640 1 -- 172.21.15.5:0/1490411804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3af804e840 con 0x7f3af814b2f0 2023-11-24T06:15:49.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.340+0000 7f3aeeffd640 1 -- 172.21.15.5:0/1490411804 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3af007aad0 con 0x7f3af814b2f0 2023-11-24T06:15:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:15:49.343 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2023-11-24T06:15:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:15:49.343 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2023-11-24T06:15:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2023-11-24T06:15:49.344 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:15:49.344 INFO:teuthology.orchestra.run.smithi005.stdout:] 2023-11-24T06:15:49.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.342+0000 7f3aff316640 1 -- 172.21.15.5:0/1490411804 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3adc06ae40 msgr2=0x7f3adc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:49.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.342+0000 7f3aff316640 1 --2- 172.21.15.5:0/1490411804 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3adc06ae40 0x7f3adc06d300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3af4001eb0 tx=0x7f3af4000ed0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.342+0000 7f3aff316640 1 -- 172.21.15.5:0/1490411804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3af814b2f0 msgr2=0x7f3af814b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:49.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.342+0000 7f3aff316640 1 --2- 172.21.15.5:0/1490411804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3af814b2f0 0x7f3af814b770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3af000e200 tx=0x7f3af0009750 comp rx=0 tx=0).stop 2023-11-24T06:15:49.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.342+0000 7f3aff316640 1 -- 172.21.15.5:0/1490411804 shutdown_connections 2023-11-24T06:15:49.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.342+0000 7f3aff316640 1 --2- 172.21.15.5:0/1490411804 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3adc06ae40 0x7f3adc06d300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.342+0000 7f3aff316640 1 --2- 172.21.15.5:0/1490411804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3af814b2f0 0x7f3af814b770 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.342+0000 7f3aff316640 1 --2- 172.21.15.5:0/1490411804 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3af8154ba0 0x7f3af80b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.342+0000 7f3aff316640 1 --2- 172.21.15.5:0/1490411804 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3ae4005180 0x7f3af814c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.342+0000 7f3aff316640 1 -- 172.21.15.5:0/1490411804 >> 172.21.15.5:0/1490411804 conn(0x7f3af806a860 msgr2=0x7f3af806ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:49.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.342+0000 7f3aff316640 1 -- 172.21.15.5:0/1490411804 shutdown_connections 2023-11-24T06:15:49.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.342+0000 7f3aff316640 1 -- 172.21.15.5:0/1490411804 wait complete. 2023-11-24T06:15:49.364 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:15:49.364 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:15:49.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.583+0000 7f637560e640 1 Processor -- start 2023-11-24T06:15:49.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.583+0000 7f637560e640 1 -- start start 2023-11-24T06:15:49.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.585+0000 7f637560e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f637009cb30 0x7f63700abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.585+0000 7f637560e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637015b730 con 0x7f63700adb60 2023-11-24T06:15:49.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.585+0000 7f637560e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63700ac740 con 0x7f637015b350 2023-11-24T06:15:49.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.585+0000 7f637560e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63700ac460 con 0x7f637009cb30 2023-11-24T06:15:49.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.585+0000 7f636f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f63700adb60 0x7f63700adfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48082/0 (socket says 172.21.15.5:48082) 2023-11-24T06:15:49.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.585+0000 7f636f7fe640 1 -- 172.21.15.5:0/3444280074 learned_addr learned my addr 172.21.15.5:0/3444280074 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:49.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.585+0000 7f636e7fc640 1 --2- 172.21.15.5:0/3444280074 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f637009cb30 0x7f63700abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 -- 172.21.15.5:0/3444280074 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195457284 0 0) 0x7f6368001490 con 0x7f63700adb60 2023-11-24T06:15:49.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 -- 172.21.15.5:0/3444280074 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6358003520 con 0x7f63700adb60 2023-11-24T06:15:49.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 -- 172.21.15.5:0/3444280074 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2468891669 0 0) 0x7f6358003520 con 0x7f63700adb60 2023-11-24T06:15:49.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 -- 172.21.15.5:0/3444280074 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f637009cb30 msgr2=0x7f63700abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:49.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 --2- 172.21.15.5:0/3444280074 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f637009cb30 0x7f63700abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 -- 172.21.15.5:0/3444280074 >> v1:172.21.15.6:6789/0 conn(0x7f637015b350 legacy=0x7f63700b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:49.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 -- 172.21.15.5:0/3444280074 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63700b2a50 con 0x7f63700adb60 2023-11-24T06:15:49.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 -- 172.21.15.5:0/3444280074 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f6368001b80 con 0x7f63700adb60 2023-11-24T06:15:49.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 -- 172.21.15.5:0/3444280074 >> v1:172.21.15.5:6789/0 conn(0x7f63700adb60 legacy=0x7f63700adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:49.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 --2- 172.21.15.5:0/3444280074 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f637009cb30 0x7f63580039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 --2- 172.21.15.5:0/3444280074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6358003f20 0x7f63580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 --2- 172.21.15.5:0/3444280074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6358004d00 0x7f6358005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 -- 172.21.15.5:0/3444280074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63700ac460 con 0x7f6358004d00 2023-11-24T06:15:49.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 -- 172.21.15.5:0/3444280074 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635800ba60 con 0x7f6358003f20 2023-11-24T06:15:49.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.586+0000 7f636dffb640 1 -- 172.21.15.5:0/3444280074 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635800bc20 con 0x7f637009cb30 2023-11-24T06:15:49.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.587+0000 7f636f7fe640 1 --2- 172.21.15.5:0/3444280074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6358004d00 0x7f6358005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.587+0000 7f636effd640 1 --2- 172.21.15.5:0/3444280074 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f637009cb30 0x7f63580039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.587+0000 7f636e7fc640 1 --2- 172.21.15.5:0/3444280074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6358003f20 0x7f63580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.587+0000 7f636f7fe640 1 -- 172.21.15.5:0/3444280074 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f637009cb30 msgr2=0x7f63580039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:49.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.587+0000 7f636f7fe640 1 --2- 172.21.15.5:0/3444280074 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f637009cb30 0x7f63580039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.587+0000 7f636f7fe640 1 -- 172.21.15.5:0/3444280074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6358003f20 msgr2=0x7f63580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:49.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.587+0000 7f636f7fe640 1 --2- 172.21.15.5:0/3444280074 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6358003f20 0x7f63580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.588+0000 7f636f7fe640 1 -- 172.21.15.5:0/3444280074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6358003520 con 0x7f6358004d00 2023-11-24T06:15:49.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.589+0000 7f636f7fe640 1 --2- 172.21.15.5:0/3444280074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6358004d00 0x7f6358005180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f636800a830 tx=0x7f63580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d94d73f7740fd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:49.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.589+0000 7f636dffb640 1 -- 172.21.15.5:0/3444280074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368034630 con 0x7f6358004d00 2023-11-24T06:15:49.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.589+0000 7f636dffb640 1 -- 172.21.15.5:0/3444280074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636802f070 con 0x7f6358004d00 2023-11-24T06:15:49.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.589+0000 7f636dffb640 1 -- 172.21.15.5:0/3444280074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368034a30 con 0x7f6358004d00 2023-11-24T06:15:49.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.590+0000 7f637560e640 1 -- 172.21.15.5:0/3444280074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6358004d00 msgr2=0x7f6358005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:49.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.590+0000 7f637560e640 1 --2- 172.21.15.5:0/3444280074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6358004d00 0x7f6358005180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f636800a830 tx=0x7f63580057f0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.590+0000 7f636f7fe640 1 -- 172.21.15.5:0/3444280074 reap_dead start 2023-11-24T06:15:49.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.590+0000 7f637560e640 1 -- 172.21.15.5:0/3444280074 shutdown_connections 2023-11-24T06:15:49.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.590+0000 7f637560e640 1 -- 172.21.15.5:0/3444280074 >> 172.21.15.5:0/3444280074 conn(0x7f63700b0ba0 msgr2=0x7f63700b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:49.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.590+0000 7f637560e640 1 -- 172.21.15.5:0/3444280074 shutdown_connections 2023-11-24T06:15:49.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.590+0000 7f637560e640 1 -- 172.21.15.5:0/3444280074 wait complete. 2023-11-24T06:15:49.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.592+0000 7f637560e640 1 Processor -- start 2023-11-24T06:15:49.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.592+0000 7f637560e640 1 -- start start 2023-11-24T06:15:49.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.593+0000 7f637560e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6358004d00 0x7f637013f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.593+0000 7f637560e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63700afcb0 0x7f63700b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.593+0000 7f637560e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f637014b980 0x7f637014be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.593+0000 7f637560e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6370149d10 con 0x7f63700afcb0 2023-11-24T06:15:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.593+0000 7f637560e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63700b0670 con 0x7f6358004d00 2023-11-24T06:15:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.593+0000 7f637560e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637014c370 con 0x7f637014b980 2023-11-24T06:15:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.593+0000 7f636e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63700afcb0 0x7f63700b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.593+0000 7f636e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63700afcb0 0x7f63700b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57508/0 (socket says 172.21.15.5:57508) 2023-11-24T06:15:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.593+0000 7f636e7fc640 1 -- 172.21.15.5:0/3817983089 learned_addr learned my addr 172.21.15.5:0/3817983089 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.593+0000 7f636effd640 1 --2- 172.21.15.5:0/3817983089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6358004d00 0x7f637013f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.593+0000 7f636f7fe640 1 --2- 172.21.15.5:0/3817983089 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f637014b980 0x7f637014be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.593+0000 7f636e7fc640 1 -- 172.21.15.5:0/3817983089 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f637014b980 msgr2=0x7f637014be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.593+0000 7f636e7fc640 1 --2- 172.21.15.5:0/3817983089 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f637014b980 0x7f637014be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.593+0000 7f636e7fc640 1 -- 172.21.15.5:0/3817983089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6358004d00 msgr2=0x7f637013f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:49.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.594+0000 7f636e7fc640 1 --2- 172.21.15.5:0/3817983089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6358004d00 0x7f637013f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:49.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.594+0000 7f636e7fc640 1 -- 172.21.15.5:0/3817983089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6370149f20 con 0x7f63700afcb0 2023-11-24T06:15:49.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.594+0000 7f636e7fc640 1 --2- 172.21.15.5:0/3817983089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63700afcb0 0x7f63700b0130 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6364011600 tx=0x7f6364011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:49.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.594+0000 7f63577fe640 1 -- 172.21.15.5:0/3817983089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636400f920 con 0x7f63700afcb0 2023-11-24T06:15:49.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.594+0000 7f63577fe640 1 -- 172.21.15.5:0/3817983089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636400fa80 con 0x7f63700afcb0 2023-11-24T06:15:49.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.594+0000 7f63577fe640 1 -- 172.21.15.5:0/3817983089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636400fdf0 con 0x7f63700afcb0 2023-11-24T06:15:49.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.594+0000 7f637560e640 1 -- 172.21.15.5:0/3817983089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637014a200 con 0x7f63700afcb0 2023-11-24T06:15:49.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.594+0000 7f637560e640 1 -- 172.21.15.5:0/3817983089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637014a7c0 con 0x7f63700afcb0 2023-11-24T06:15:49.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.596+0000 7f637560e640 1 -- 172.21.15.5:0/3817983089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637009ba10 con 0x7f63700afcb0 2023-11-24T06:15:49.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.596+0000 7f63577fe640 1 -- 172.21.15.5:0/3817983089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6364015460 con 0x7f63700afcb0 2023-11-24T06:15:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.598+0000 7f63577fe640 1 --2- 172.21.15.5:0/3817983089 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f635c06ae90 0x7f635c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.598+0000 7f63577fe640 1 -- 172.21.15.5:0/3817983089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6364089b20 con 0x7f63700afcb0 2023-11-24T06:15:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.598+0000 7f636effd640 1 --2- 172.21.15.5:0/3817983089 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f635c06ae90 0x7f635c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.599+0000 7f636effd640 1 --2- 172.21.15.5:0/3817983089 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f635c06ae90 0x7f635c06d350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6360005590 tx=0x7f6360009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.602+0000 7f63577fe640 1 -- 172.21.15.5:0/3817983089 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6364054f90 con 0x7f63700afcb0 2023-11-24T06:15:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:49.796+0000 7f637560e640 1 -- 172.21.15.5:0/3817983089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f637009d200 con 0x7f63700afcb0 2023-11-24T06:15:50.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.108+0000 7f63577fe640 1 -- 172.21.15.5:0/3817983089 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f63640585b0 con 0x7f63700afcb0 2023-11-24T06:15:50.112 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2023-11-24T06:15:50.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.113+0000 7f637560e640 1 -- 172.21.15.5:0/3817983089 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f635c06ae90 msgr2=0x7f635c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:50.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.113+0000 7f637560e640 1 --2- 172.21.15.5:0/3817983089 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f635c06ae90 0x7f635c06d350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6360005590 tx=0x7f6360009040 comp rx=0 tx=0).stop 2023-11-24T06:15:50.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.113+0000 7f637560e640 1 -- 172.21.15.5:0/3817983089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63700afcb0 msgr2=0x7f63700b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:50.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.113+0000 7f637560e640 1 --2- 172.21.15.5:0/3817983089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63700afcb0 0x7f63700b0130 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6364011600 tx=0x7f6364011ac0 comp rx=0 tx=0).stop 2023-11-24T06:15:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.114+0000 7f637560e640 1 -- 172.21.15.5:0/3817983089 shutdown_connections 2023-11-24T06:15:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.114+0000 7f637560e640 1 --2- 172.21.15.5:0/3817983089 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f635c06ae90 0x7f635c06d350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.114+0000 7f637560e640 1 --2- 172.21.15.5:0/3817983089 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f637014b980 0x7f637014be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.114+0000 7f637560e640 1 --2- 172.21.15.5:0/3817983089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63700afcb0 0x7f63700b0130 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.114+0000 7f637560e640 1 --2- 172.21.15.5:0/3817983089 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6358004d00 0x7f637013f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.114+0000 7f637560e640 1 -- 172.21.15.5:0/3817983089 >> 172.21.15.5:0/3817983089 conn(0x7f63700b0ba0 msgr2=0x7f6370159270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:50.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.114+0000 7f637560e640 1 -- 172.21.15.5:0/3817983089 shutdown_connections 2023-11-24T06:15:50.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.115+0000 7f637560e640 1 -- 172.21.15.5:0/3817983089 wait complete. 2023-11-24T06:15:50.146 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:15:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.350+0000 7fb935d89640 1 Processor -- start 2023-11-24T06:15:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.350+0000 7fb935d89640 1 -- start start 2023-11-24T06:15:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.352+0000 7fb935d89640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb930162600 0x7fb930162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.352+0000 7fb935d89640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb93006c3c0 0x7fb93006c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.352+0000 7fb935d89640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb930162fa0 con 0x7fb930162220 2023-11-24T06:15:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.352+0000 7fb935d89640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb93006cd90 con 0x7fb93006c3c0 2023-11-24T06:15:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.352+0000 7fb935d89640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9300706a0 con 0x7fb930162600 2023-11-24T06:15:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.352+0000 7fb92f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb930162220 0x7fb9300b1ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48096/0 (socket says 172.21.15.5:48096) 2023-11-24T06:15:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.353+0000 7fb92f7fe640 1 -- 172.21.15.5:0/2597283168 learned_addr learned my addr 172.21.15.5:0/2597283168 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.353+0000 7fb92ffff640 1 --2- 172.21.15.5:0/2597283168 >> v2:172.21.15.5:3300/0 conn(0x7fb93006c3c0 0x7fb93006c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.353+0000 7fb92effd640 1 --2- 172.21.15.5:0/2597283168 >> v2:172.21.15.6:3300/0 conn(0x7fb930162600 0x7fb930162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.353+0000 7fb92ffff640 1 -- 172.21.15.5:0/2597283168 >> v2:172.21.15.6:3300/0 conn(0x7fb930162600 msgr2=0x7fb930162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.353+0000 7fb92e7fc640 1 -- 172.21.15.5:0/2597283168 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436533534 0 0) 0x7fb930162fa0 con 0x7fb930162220 2023-11-24T06:15:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.353+0000 7fb92ffff640 1 --2- 172.21.15.5:0/2597283168 >> v2:172.21.15.6:3300/0 conn(0x7fb930162600 0x7fb930162a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.353+0000 7fb92ffff640 1 -- 172.21.15.5:0/2597283168 >> v1:172.21.15.5:6789/0 conn(0x7fb930162220 legacy=0x7fb9300b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.353+0000 7fb92ffff640 1 -- 172.21.15.5:0/2597283168 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930070c60 con 0x7fb93006c3c0 2023-11-24T06:15:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.353+0000 7fb92effd640 1 --2- 172.21.15.5:0/2597283168 >> v2:172.21.15.6:3300/0 conn(0x7fb930162600 0x7fb930162a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:15:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.354+0000 7fb92ffff640 1 --2- 172.21.15.5:0/2597283168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93006c3c0 0x7fb93006c850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb93006d8f0 tx=0x7fb92802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c1561fb8f59d38 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.355+0000 7fb92e7fc640 1 -- 172.21.15.5:0/2597283168 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92802cc70 con 0x7fb93006c3c0 2023-11-24T06:15:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.355+0000 7fb92e7fc640 1 -- 172.21.15.5:0/2597283168 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92802cdd0 con 0x7fb93006c3c0 2023-11-24T06:15:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.355+0000 7fb92e7fc640 1 -- 172.21.15.5:0/2597283168 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928034660 con 0x7fb93006c3c0 2023-11-24T06:15:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.355+0000 7fb935d89640 1 -- 172.21.15.5:0/2597283168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93006c3c0 msgr2=0x7fb93006c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.355+0000 7fb935d89640 1 --2- 172.21.15.5:0/2597283168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93006c3c0 0x7fb93006c850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb93006d8f0 tx=0x7fb92802bec0 comp rx=0 tx=0).stop 2023-11-24T06:15:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.355+0000 7fb935d89640 1 -- 172.21.15.5:0/2597283168 shutdown_connections 2023-11-24T06:15:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.355+0000 7fb935d89640 1 --2- 172.21.15.5:0/2597283168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93006c3c0 0x7fb93006c850 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.355+0000 7fb935d89640 1 --2- 172.21.15.5:0/2597283168 >> v2:172.21.15.6:3300/0 conn(0x7fb930162600 0x7fb930162a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.355+0000 7fb935d89640 1 -- 172.21.15.5:0/2597283168 >> 172.21.15.5:0/2597283168 conn(0x7fb9300b0ba0 msgr2=0x7fb9300b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.356+0000 7fb935d89640 1 -- 172.21.15.5:0/2597283168 shutdown_connections 2023-11-24T06:15:50.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.356+0000 7fb935d89640 1 -- 172.21.15.5:0/2597283168 wait complete. 2023-11-24T06:15:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.357+0000 7fb935d89640 1 Processor -- start 2023-11-24T06:15:50.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.357+0000 7fb935d89640 1 -- start start 2023-11-24T06:15:50.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.358+0000 7fb935d89640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb93006c3c0 0x7fb930164ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.358+0000 7fb935d89640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb930162220 0x7fb930152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.358+0000 7fb935d89640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb930162600 0x7fb9300aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.358+0000 7fb935d89640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9301655e0 con 0x7fb930162600 2023-11-24T06:15:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.358+0000 7fb935d89640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb930099b80 con 0x7fb930162220 2023-11-24T06:15:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.358+0000 7fb935d89640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb930099e60 con 0x7fb93006c3c0 2023-11-24T06:15:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.359+0000 7fb92ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb930162600 0x7fb9300aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.359+0000 7fb92ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb930162600 0x7fb9300aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57524/0 (socket says 172.21.15.5:57524) 2023-11-24T06:15:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.359+0000 7fb92ffff640 1 -- 172.21.15.5:0/3075891064 learned_addr learned my addr 172.21.15.5:0/3075891064 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.359+0000 7fb92effd640 1 --2- 172.21.15.5:0/3075891064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb930162220 0x7fb930152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.359+0000 7fb92f7fe640 1 --2- 172.21.15.5:0/3075891064 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb93006c3c0 0x7fb930164ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:50.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.359+0000 7fb92ffff640 1 -- 172.21.15.5:0/3075891064 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb93006c3c0 msgr2=0x7fb930164ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.359+0000 7fb92ffff640 1 --2- 172.21.15.5:0/3075891064 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb93006c3c0 0x7fb930164ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.359+0000 7fb92ffff640 1 -- 172.21.15.5:0/3075891064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb930162220 msgr2=0x7fb930152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.359+0000 7fb92ffff640 1 --2- 172.21.15.5:0/3075891064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb930162220 0x7fb930152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.359+0000 7fb92ffff640 1 -- 172.21.15.5:0/3075891064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb928009a70 con 0x7fb930162600 2023-11-24T06:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.360+0000 7fb92ffff640 1 --2- 172.21.15.5:0/3075891064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb930162600 0x7fb9300aced0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb92802c4d0 tx=0x7fb928004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.360+0000 7fb917fff640 1 -- 172.21.15.5:0/3075891064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92803e610 con 0x7fb930162600 2023-11-24T06:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.360+0000 7fb917fff640 1 -- 172.21.15.5:0/3075891064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92803e770 con 0x7fb930162600 2023-11-24T06:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.360+0000 7fb917fff640 1 -- 172.21.15.5:0/3075891064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92803eb20 con 0x7fb930162600 2023-11-24T06:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.360+0000 7fb935d89640 1 -- 172.21.15.5:0/3075891064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93009a140 con 0x7fb930162600 2023-11-24T06:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.360+0000 7fb935d89640 1 -- 172.21.15.5:0/3075891064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb930153190 con 0x7fb930162600 2023-11-24T06:15:50.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.361+0000 7fb935d89640 1 -- 172.21.15.5:0/3075891064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9301533b0 con 0x7fb930162600 2023-11-24T06:15:50.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.363+0000 7fb917fff640 1 -- 172.21.15.5:0/3075891064 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb92803b050 con 0x7fb930162600 2023-11-24T06:15:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.364+0000 7fb917fff640 1 --2- 172.21.15.5:0/3075891064 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb91c06ae70 0x7fb91c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.364+0000 7fb917fff640 1 -- 172.21.15.5:0/3075891064 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb9280abc50 con 0x7fb930162600 2023-11-24T06:15:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.364+0000 7fb92f7fe640 1 --2- 172.21.15.5:0/3075891064 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb91c06ae70 0x7fb91c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.365+0000 7fb92f7fe640 1 --2- 172.21.15.5:0/3075891064 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb91c06ae70 0x7fb91c06d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb920000f60 tx=0x7fb920007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:50.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.369+0000 7fb917fff640 1 -- 172.21.15.5:0/3075891064 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb928076f60 con 0x7fb930162600 2023-11-24T06:15:50.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:50.562+0000 7fb935d89640 1 -- 172.21.15.5:0/3075891064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb93009d200 con 0x7fb930162600 2023-11-24T06:15:51.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.125+0000 7fb917fff640 1 -- 172.21.15.5:0/3075891064 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb9280390d0 con 0x7fb930162600 2023-11-24T06:15:51.128 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2023-11-24T06:15:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.128+0000 7fb935d89640 1 -- 172.21.15.5:0/3075891064 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb91c06ae70 msgr2=0x7fb91c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.128+0000 7fb935d89640 1 --2- 172.21.15.5:0/3075891064 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb91c06ae70 0x7fb91c06d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb920000f60 tx=0x7fb920007c00 comp rx=0 tx=0).stop 2023-11-24T06:15:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.128+0000 7fb935d89640 1 -- 172.21.15.5:0/3075891064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb930162600 msgr2=0x7fb9300aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.128+0000 7fb935d89640 1 --2- 172.21.15.5:0/3075891064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb930162600 0x7fb9300aced0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb92802c4d0 tx=0x7fb928004a40 comp rx=0 tx=0).stop 2023-11-24T06:15:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.129+0000 7fb935d89640 1 -- 172.21.15.5:0/3075891064 shutdown_connections 2023-11-24T06:15:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.129+0000 7fb935d89640 1 --2- 172.21.15.5:0/3075891064 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb91c06ae70 0x7fb91c06d330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.129+0000 7fb935d89640 1 --2- 172.21.15.5:0/3075891064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb930162600 0x7fb9300aced0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.129+0000 7fb935d89640 1 --2- 172.21.15.5:0/3075891064 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb930162220 0x7fb930152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.129+0000 7fb935d89640 1 --2- 172.21.15.5:0/3075891064 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb93006c3c0 0x7fb930164ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.129+0000 7fb935d89640 1 -- 172.21.15.5:0/3075891064 >> 172.21.15.5:0/3075891064 conn(0x7fb9300b0ba0 msgr2=0x7fb9300aef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.130+0000 7fb935d89640 1 -- 172.21.15.5:0/3075891064 shutdown_connections 2023-11-24T06:15:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.130+0000 7fb935d89640 1 -- 172.21.15.5:0/3075891064 wait complete. 2023-11-24T06:15:51.166 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:15:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.355+0000 7f2d477be640 1 Processor -- start 2023-11-24T06:15:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.355+0000 7f2d477be640 1 -- start start 2023-11-24T06:15:51.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.357+0000 7f2d477be640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2d400cd150 0x7f2d4006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:51.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.357+0000 7f2d477be640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2d400702a0 0x7f2d4006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:51.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.357+0000 7f2d477be640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d4006ec60 0x7f2d40160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:51.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.357+0000 7f2d477be640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d400cd530 con 0x7f2d400cd150 2023-11-24T06:15:51.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.357+0000 7f2d477be640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d40142d50 con 0x7f2d400702a0 2023-11-24T06:15:51.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.357+0000 7f2d477be640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400706c0 con 0x7f2d4006ec60 2023-11-24T06:15:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.357+0000 7f2d4555a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2d400cd150 0x7f2d4006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.357+0000 7f2d4555a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2d400cd150 0x7f2d4006fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57540/0 (socket says 172.21.15.5:57540) 2023-11-24T06:15:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.357+0000 7f2d4555a640 1 -- 172.21.15.5:0/3004641541 learned_addr learned my addr 172.21.15.5:0/3004641541 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.358+0000 7f2d44d59640 1 --2- 172.21.15.5:0/3004641541 >> v2:172.21.15.6:3300/0 conn(0x7f2d400702a0 0x7f2d4006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.358+0000 7f2d45d5b640 1 --2- 172.21.15.5:0/3004641541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d4006ec60 0x7f2d40160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.358+0000 7f2d4555a640 1 -- 172.21.15.5:0/3004641541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d4006ec60 msgr2=0x7f2d40160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.358+0000 7f2d4555a640 1 --2- 172.21.15.5:0/3004641541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d4006ec60 0x7f2d40160fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:51.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.358+0000 7f2d4555a640 1 -- 172.21.15.5:0/3004641541 >> v2:172.21.15.6:3300/0 conn(0x7f2d400702a0 msgr2=0x7f2d4006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:51.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.358+0000 7f2d4555a640 1 --2- 172.21.15.5:0/3004641541 >> v2:172.21.15.6:3300/0 conn(0x7f2d400702a0 0x7f2d4006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:51.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.358+0000 7f2d4555a640 1 -- 172.21.15.5:0/3004641541 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4006f430 con 0x7f2d400cd150 2023-11-24T06:15:51.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.358+0000 7f2d44d59640 1 --2- 172.21.15.5:0/3004641541 >> v2:172.21.15.6:3300/0 conn(0x7f2d400702a0 0x7f2d4006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:15:51.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.360+0000 7f2d4555a640 1 --2- 172.21.15.5:0/3004641541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d400cd150 0x7f2d4006fd60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2d40147310 tx=0x7f2d3c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14deffc6258fbac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:51.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.360+0000 7f2d37fff640 1 -- 172.21.15.5:0/3004641541 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c02c9f0 con 0x7f2d400cd150 2023-11-24T06:15:51.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.360+0000 7f2d37fff640 1 -- 172.21.15.5:0/3004641541 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d3c02cb50 con 0x7f2d400cd150 2023-11-24T06:15:51.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.360+0000 7f2d37fff640 1 -- 172.21.15.5:0/3004641541 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c0343f0 con 0x7f2d400cd150 2023-11-24T06:15:51.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.361+0000 7f2d477be640 1 -- 172.21.15.5:0/3004641541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d400cd150 msgr2=0x7f2d4006fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:51.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.361+0000 7f2d477be640 1 --2- 172.21.15.5:0/3004641541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d400cd150 0x7f2d4006fd60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2d40147310 tx=0x7f2d3c02be50 comp rx=0 tx=0).stop 2023-11-24T06:15:51.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.361+0000 7f2d477be640 1 -- 172.21.15.5:0/3004641541 shutdown_connections 2023-11-24T06:15:51.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.361+0000 7f2d477be640 1 --2- 172.21.15.5:0/3004641541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d4006ec60 0x7f2d40160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:51.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.361+0000 7f2d477be640 1 --2- 172.21.15.5:0/3004641541 >> v2:172.21.15.6:3300/0 conn(0x7f2d400702a0 0x7f2d4006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:51.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.361+0000 7f2d477be640 1 --2- 172.21.15.5:0/3004641541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d400cd150 0x7f2d4006fd60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:51.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.361+0000 7f2d477be640 1 -- 172.21.15.5:0/3004641541 >> 172.21.15.5:0/3004641541 conn(0x7f2d400cdb20 msgr2=0x7f2d400cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:51.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.361+0000 7f2d477be640 1 -- 172.21.15.5:0/3004641541 shutdown_connections 2023-11-24T06:15:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.361+0000 7f2d477be640 1 -- 172.21.15.5:0/3004641541 wait complete. 2023-11-24T06:15:51.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.363+0000 7f2d477be640 1 Processor -- start 2023-11-24T06:15:51.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.363+0000 7f2d477be640 1 -- start start 2023-11-24T06:15:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.365+0000 7f2d477be640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d4006ec60 0x7f2d401f2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.365+0000 7f2d477be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d400702a0 0x7f2d400b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.365+0000 7f2d477be640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d400cd150 0x7f2d400c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.365+0000 7f2d477be640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d401f3ed0 con 0x7f2d400702a0 2023-11-24T06:15:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.365+0000 7f2d477be640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d400b4790 con 0x7f2d4006ec60 2023-11-24T06:15:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.365+0000 7f2d477be640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d401f6770 con 0x7f2d400cd150 2023-11-24T06:15:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.365+0000 7f2d44d59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d400702a0 0x7f2d400b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.365+0000 7f2d44d59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d400702a0 0x7f2d400b4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57544/0 (socket says 172.21.15.5:57544) 2023-11-24T06:15:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.365+0000 7f2d44d59640 1 -- 172.21.15.5:0/1241677170 learned_addr learned my addr 172.21.15.5:0/1241677170 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.365+0000 7f2d4555a640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d4006ec60 0x7f2d401f2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.365+0000 7f2d45d5b640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d400cd150 0x7f2d400c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.366+0000 7f2d44d59640 1 -- 172.21.15.5:0/1241677170 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d400cd150 msgr2=0x7f2d400c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.366+0000 7f2d44d59640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d400cd150 0x7f2d400c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.366+0000 7f2d44d59640 1 -- 172.21.15.5:0/1241677170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d4006ec60 msgr2=0x7f2d401f2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.366+0000 7f2d44d59640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d4006ec60 0x7f2d401f2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.366+0000 7f2d44d59640 1 -- 172.21.15.5:0/1241677170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3c009b30 con 0x7f2d400702a0 2023-11-24T06:15:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.366+0000 7f2d45d5b640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d400cd150 0x7f2d400c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:15:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.366+0000 7f2d44d59640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d400702a0 0x7f2d400b4250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000d800 tx=0x7f2d3000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.366+0000 7f2d35ffb640 1 -- 172.21.15.5:0/1241677170 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30009640 con 0x7f2d400702a0 2023-11-24T06:15:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.366+0000 7f2d35ffb640 1 -- 172.21.15.5:0/1241677170 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d300097a0 con 0x7f2d400702a0 2023-11-24T06:15:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.366+0000 7f2d35ffb640 1 -- 172.21.15.5:0/1241677170 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d30009b50 con 0x7f2d400702a0 2023-11-24T06:15:51.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.366+0000 7f2d477be640 1 -- 172.21.15.5:0/1241677170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d401f3630 con 0x7f2d400702a0 2023-11-24T06:15:51.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.366+0000 7f2d477be640 1 -- 172.21.15.5:0/1241677170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d401f3bf0 con 0x7f2d400702a0 2023-11-24T06:15:51.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.368+0000 7f2d477be640 1 -- 172.21.15.5:0/1241677170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d401f6540 con 0x7f2d400702a0 2023-11-24T06:15:51.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.369+0000 7f2d35ffb640 1 -- 172.21.15.5:0/1241677170 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2d30009cb0 con 0x7f2d400702a0 2023-11-24T06:15:51.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.370+0000 7f2d35ffb640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2d2406ae40 0x7f2d2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.370+0000 7f2d35ffb640 1 -- 172.21.15.5:0/1241677170 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2d30089a70 con 0x7f2d400702a0 2023-11-24T06:15:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.370+0000 7f2d4555a640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2d2406ae40 0x7f2d2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.371+0000 7f2d4555a640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2d2406ae40 0x7f2d2406d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c035000 tx=0x7f2d3c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:51.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.375+0000 7f2d35ffb640 1 -- 172.21.15.5:0/1241677170 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d401f6540 con 0x7f2d400702a0 2023-11-24T06:15:51.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:51.609+0000 7f2d477be640 1 -- 172.21.15.5:0/1241677170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2d401f6540 con 0x7f2d400702a0 2023-11-24T06:15:52.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.128+0000 7f2d35ffb640 1 -- 172.21.15.5:0/1241677170 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2d401f6540 con 0x7f2d400702a0 2023-11-24T06:15:52.131 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2023-11-24T06:15:52.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.132+0000 7f2d477be640 1 -- 172.21.15.5:0/1241677170 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2d2406ae40 msgr2=0x7f2d2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:52.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.132+0000 7f2d477be640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2d2406ae40 0x7f2d2406d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c035000 tx=0x7f2d3c030000 comp rx=0 tx=0).stop 2023-11-24T06:15:52.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.132+0000 7f2d477be640 1 -- 172.21.15.5:0/1241677170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d400702a0 msgr2=0x7f2d400b4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:52.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.132+0000 7f2d477be640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d400702a0 0x7f2d400b4250 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d3000d800 tx=0x7f2d3000dcc0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.132+0000 7f2d477be640 1 -- 172.21.15.5:0/1241677170 shutdown_connections 2023-11-24T06:15:52.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.132+0000 7f2d477be640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2d2406ae40 0x7f2d2406d300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.132+0000 7f2d477be640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2d400cd150 0x7f2d400c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.132+0000 7f2d477be640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d400702a0 0x7f2d400b4250 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.132+0000 7f2d477be640 1 --2- 172.21.15.5:0/1241677170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2d4006ec60 0x7f2d401f2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.132+0000 7f2d477be640 1 -- 172.21.15.5:0/1241677170 >> 172.21.15.5:0/1241677170 conn(0x7f2d400cdb20 msgr2=0x7f2d400d1860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:52.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.132+0000 7f2d477be640 1 -- 172.21.15.5:0/1241677170 shutdown_connections 2023-11-24T06:15:52.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.133+0000 7f2d477be640 1 -- 172.21.15.5:0/1241677170 wait complete. 2023-11-24T06:15:52.162 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:15:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.355+0000 7fbbc5fdc640 1 Processor -- start 2023-11-24T06:15:52.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.355+0000 7fbbc5fdc640 1 -- start start 2023-11-24T06:15:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.356+0000 7fbbc5fdc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbbc006d520 0x7fbbc006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.357+0000 7fbbc5fdc640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbbc00ad720 0x7fbbc015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.357+0000 7fbbc5fdc640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc015c360 con 0x7fbbc00abff0 2023-11-24T06:15:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.357+0000 7fbbc5fdc640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc00b41a0 con 0x7fbbc006d520 2023-11-24T06:15:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.357+0000 7fbbc5fdc640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc006de40 con 0x7fbbc00ad720 2023-11-24T06:15:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.357+0000 7fbbbf7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbbc006d520 0x7fbbc006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.357+0000 7fbbbf7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbbc006d520 0x7fbbc006d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57556/0 (socket says 172.21.15.5:57556) 2023-11-24T06:15:52.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.357+0000 7fbbbf7fe640 1 -- 172.21.15.5:0/614580423 learned_addr learned my addr 172.21.15.5:0/614580423 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:52.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.357+0000 7fbbbeffd640 1 --2- 172.21.15.5:0/614580423 >> v2:172.21.15.6:3300/0 conn(0x7fbbc00ad720 0x7fbbc015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:52.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.358+0000 7fbbbf7fe640 1 -- 172.21.15.5:0/614580423 >> v2:172.21.15.6:3300/0 conn(0x7fbbc00ad720 msgr2=0x7fbbc015ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:52.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.358+0000 7fbbbf7fe640 1 --2- 172.21.15.5:0/614580423 >> v2:172.21.15.6:3300/0 conn(0x7fbbc00ad720 0x7fbbc015ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.358+0000 7fbbbf7fe640 1 -- 172.21.15.5:0/614580423 >> v1:172.21.15.6:6789/0 conn(0x7fbbc00abff0 legacy=0x7fbbc015f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:52.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.358+0000 7fbbbf7fe640 1 -- 172.21.15.5:0/614580423 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc015ba30 con 0x7fbbc006d520 2023-11-24T06:15:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.359+0000 7fbbbf7fe640 1 --2- 172.21.15.5:0/614580423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc006d520 0x7fbbc006d900 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00adb00 tx=0x7fbbb002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f1c1ba6607d740 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.359+0000 7fbbbe7fc640 1 -- 172.21.15.5:0/614580423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb002ca80 con 0x7fbbc006d520 2023-11-24T06:15:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.359+0000 7fbbbe7fc640 1 -- 172.21.15.5:0/614580423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb002cbe0 con 0x7fbbc006d520 2023-11-24T06:15:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.359+0000 7fbbbe7fc640 1 -- 172.21.15.5:0/614580423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb00343f0 con 0x7fbbc006d520 2023-11-24T06:15:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.360+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/614580423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc006d520 msgr2=0x7fbbc006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.360+0000 7fbbc5fdc640 1 --2- 172.21.15.5:0/614580423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc006d520 0x7fbbc006d900 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00adb00 tx=0x7fbbb002bee0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.360+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/614580423 shutdown_connections 2023-11-24T06:15:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.360+0000 7fbbc5fdc640 1 --2- 172.21.15.5:0/614580423 >> v2:172.21.15.6:3300/0 conn(0x7fbbc00ad720 0x7fbbc015ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.360+0000 7fbbc5fdc640 1 --2- 172.21.15.5:0/614580423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc006d520 0x7fbbc006d900 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.360+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/614580423 >> 172.21.15.5:0/614580423 conn(0x7fbbc00b0d80 msgr2=0x7fbbc00b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.360+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/614580423 shutdown_connections 2023-11-24T06:15:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.360+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/614580423 wait complete. 2023-11-24T06:15:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.362+0000 7fbbc5fdc640 1 Processor -- start 2023-11-24T06:15:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.362+0000 7fbbc5fdc640 1 -- start start 2023-11-24T06:15:52.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.363+0000 7fbbc5fdc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc006d520 0x7fbbc014a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:52.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.363+0000 7fbbc5fdc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbc00abff0 0x7fbbc0071b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:52.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.363+0000 7fbbc5fdc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbbc00ad720 0x7fbbc00720d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:52.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.363+0000 7fbbc5fdc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00ca710 con 0x7fbbc006d520 2023-11-24T06:15:52.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.363+0000 7fbbc5fdc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc014a820 con 0x7fbbc00abff0 2023-11-24T06:15:52.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.363+0000 7fbbc5fdc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc00c9b00 con 0x7fbbc00ad720 2023-11-24T06:15:52.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.363+0000 7fbbbf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc006d520 0x7fbbc014a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:52.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.363+0000 7fbbbf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc006d520 0x7fbbc014a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57558/0 (socket says 172.21.15.5:57558) 2023-11-24T06:15:52.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.363+0000 7fbbbf7fe640 1 -- 172.21.15.5:0/2200020056 learned_addr learned my addr 172.21.15.5:0/2200020056 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:52.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.363+0000 7fbbbeffd640 1 --2- 172.21.15.5:0/2200020056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbc00abff0 0x7fbbc0071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:52.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.364+0000 7fbbbf7fe640 1 -- 172.21.15.5:0/2200020056 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbbc00ad720 msgr2=0x7fbbc00720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:52.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.364+0000 7fbbbf7fe640 1 --2- 172.21.15.5:0/2200020056 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbbc00ad720 0x7fbbc00720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.364+0000 7fbbbf7fe640 1 -- 172.21.15.5:0/2200020056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbc00abff0 msgr2=0x7fbbc0071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:52.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.364+0000 7fbbbf7fe640 1 --2- 172.21.15.5:0/2200020056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbc00abff0 0x7fbbc0071b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.364+0000 7fbbbf7fe640 1 -- 172.21.15.5:0/2200020056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb0009b30 con 0x7fbbc006d520 2023-11-24T06:15:52.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.364+0000 7fbbbeffd640 1 --2- 172.21.15.5:0/2200020056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbc00abff0 0x7fbbc0071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:15:52.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.364+0000 7fbbbf7fe640 1 --2- 172.21.15.5:0/2200020056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc006d520 0x7fbbc014a2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0036040 tx=0x7fbbb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:52.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.364+0000 7fbba7fff640 1 -- 172.21.15.5:0/2200020056 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0009510 con 0x7fbbc006d520 2023-11-24T06:15:52.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.364+0000 7fbba7fff640 1 -- 172.21.15.5:0/2200020056 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb003e450 con 0x7fbbc006d520 2023-11-24T06:15:52.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.364+0000 7fbba7fff640 1 -- 172.21.15.5:0/2200020056 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb003e800 con 0x7fbbc006d520 2023-11-24T06:15:52.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.364+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/2200020056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc00c9de0 con 0x7fbbc006d520 2023-11-24T06:15:52.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.365+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/2200020056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc00ca3a0 con 0x7fbbc006d520 2023-11-24T06:15:52.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.366+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/2200020056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc00ca560 con 0x7fbbc006d520 2023-11-24T06:15:52.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.367+0000 7fbba7fff640 1 -- 172.21.15.5:0/2200020056 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbbb002cd50 con 0x7fbbc006d520 2023-11-24T06:15:52.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.367+0000 7fbba7fff640 1 --2- 172.21.15.5:0/2200020056 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbbac06ae40 0x7fbbac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:52.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.368+0000 7fbba7fff640 1 -- 172.21.15.5:0/2200020056 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbb00ac2c0 con 0x7fbbc006d520 2023-11-24T06:15:52.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.368+0000 7fbbbeffd640 1 --2- 172.21.15.5:0/2200020056 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbbac06ae40 0x7fbbac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:52.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.368+0000 7fbbbeffd640 1 --2- 172.21.15.5:0/2200020056 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbbac06ae40 0x7fbbac06d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4006fd0 tx=0x7fbbb4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.372+0000 7fbba7fff640 1 -- 172.21.15.5:0/2200020056 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbbb0077240 con 0x7fbbc006d520 2023-11-24T06:15:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.567+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/2200020056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbc0074770 con 0x7fbbc006d520 2023-11-24T06:15:52.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.569+0000 7fbba7fff640 1 -- 172.21.15.5:0/2200020056 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16374 (secure 0 0 0) 0x7fbbb007a860 con 0x7fbbc006d520 2023-11-24T06:15:52.573 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:15:52.573 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:51.610924+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:15:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.572+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/2200020056 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbbac06ae40 msgr2=0x7fbbac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:52.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.572+0000 7fbbc5fdc640 1 --2- 172.21.15.5:0/2200020056 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbbac06ae40 0x7fbbac06d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4006fd0 tx=0x7fbbb4008040 comp rx=0 tx=0).stop 2023-11-24T06:15:52.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.572+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/2200020056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc006d520 msgr2=0x7fbbc014a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:52.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.572+0000 7fbbc5fdc640 1 --2- 172.21.15.5:0/2200020056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc006d520 0x7fbbc014a2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0036040 tx=0x7fbbb0004a40 comp rx=0 tx=0).stop 2023-11-24T06:15:52.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.572+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/2200020056 shutdown_connections 2023-11-24T06:15:52.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.572+0000 7fbbc5fdc640 1 --2- 172.21.15.5:0/2200020056 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbbac06ae40 0x7fbbac06d300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.573+0000 7fbbc5fdc640 1 --2- 172.21.15.5:0/2200020056 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbbc00ad720 0x7fbbc00720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.573+0000 7fbbc5fdc640 1 --2- 172.21.15.5:0/2200020056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbbc00abff0 0x7fbbc0071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.573+0000 7fbbc5fdc640 1 --2- 172.21.15.5:0/2200020056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc006d520 0x7fbbc014a2e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.573+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/2200020056 >> 172.21.15.5:0/2200020056 conn(0x7fbbc00b0d80 msgr2=0x7fbbc00b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:52.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.573+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/2200020056 shutdown_connections 2023-11-24T06:15:52.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.573+0000 7fbbc5fdc640 1 -- 172.21.15.5:0/2200020056 wait complete. 2023-11-24T06:15:52.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:15:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.811+0000 7fbc25d01640 1 Processor -- start 2023-11-24T06:15:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.811+0000 7fbc25d01640 1 -- start start 2023-11-24T06:15:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.813+0000 7fbc25d01640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbc2006ef40 0x7fbc2015e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.813+0000 7fbc25d01640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc201464f0 con 0x7fbc20146110 2023-11-24T06:15:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.813+0000 7fbc25d01640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2006c710 con 0x7fbc2006eab0 2023-11-24T06:15:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.813+0000 7fbc25d01640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc20146e90 con 0x7fbc2006ef40 2023-11-24T06:15:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.813+0000 7fbc1f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbc20146110 0x7fbc201618d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:48106/0 (socket says 172.21.15.5:48106) 2023-11-24T06:15:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.813+0000 7fbc1f7fe640 1 -- 172.21.15.5:0/2516183265 learned_addr learned my addr 172.21.15.5:0/2516183265 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.813+0000 7fbc1effd640 1 -- 172.21.15.5:0/2516183265 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657303965 0 0) 0x7fbc10001110 con 0x7fbc20146110 2023-11-24T06:15:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.813+0000 7fbc1ffff640 1 --2- 172.21.15.5:0/2516183265 >> v2:172.21.15.6:3300/0 conn(0x7fbc2006ef40 0x7fbc2015e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.813+0000 7fbc1effd640 1 -- 172.21.15.5:0/2516183265 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc0c003520 con 0x7fbc20146110 2023-11-24T06:15:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.814+0000 7fbc1effd640 1 -- 172.21.15.5:0/2516183265 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 551316991 0 0) 0x7fbc0c003520 con 0x7fbc20146110 2023-11-24T06:15:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.814+0000 7fbc1effd640 1 -- 172.21.15.5:0/2516183265 >> v2:172.21.15.6:3300/0 conn(0x7fbc2006ef40 msgr2=0x7fbc2015e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.814+0000 7fbc1effd640 1 --2- 172.21.15.5:0/2516183265 >> v2:172.21.15.6:3300/0 conn(0x7fbc2006ef40 0x7fbc2015e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.814+0000 7fbc1effd640 1 -- 172.21.15.5:0/2516183265 >> v1:172.21.15.6:6789/0 conn(0x7fbc2006eab0 legacy=0x7fbc2015d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.814+0000 7fbc1effd640 1 -- 172.21.15.5:0/2516183265 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2006f3b0 con 0x7fbc20146110 2023-11-24T06:15:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.814+0000 7fbc1effd640 1 -- 172.21.15.5:0/2516183265 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fbc10001800 con 0x7fbc20146110 2023-11-24T06:15:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.814+0000 7fbc1effd640 1 -- 172.21.15.5:0/2516183265 >> v1:172.21.15.5:6789/0 conn(0x7fbc20146110 legacy=0x7fbc201618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.814+0000 7fbc1effd640 1 --2- 172.21.15.5:0/2516183265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc0c003970 0x7fbc0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.814+0000 7fbc1effd640 1 --2- 172.21.15.5:0/2516183265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc0c0047c0 0x7fbc0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.814+0000 7fbc1effd640 1 --2- 172.21.15.5:0/2516183265 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc0c005180 0x7fbc0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.814+0000 7fbc1effd640 1 -- 172.21.15.5:0/2516183265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc20146e90 con 0x7fbc0c003970 2023-11-24T06:15:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.814+0000 7fbc1effd640 1 -- 172.21.15.5:0/2516183265 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c00c020 con 0x7fbc0c0047c0 2023-11-24T06:15:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.814+0000 7fbc1effd640 1 -- 172.21.15.5:0/2516183265 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c00c1e0 con 0x7fbc0c005180 2023-11-24T06:15:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.814+0000 7fbc1f7fe640 1 --2- 172.21.15.5:0/2516183265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc0c003970 0x7fbc0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.815+0000 7fbc1f7fe640 1 -- 172.21.15.5:0/2516183265 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc0c005180 msgr2=0x7fbc0c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.815+0000 7fbc1f7fe640 1 --2- 172.21.15.5:0/2516183265 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc0c005180 0x7fbc0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.815+0000 7fbc1f7fe640 1 -- 172.21.15.5:0/2516183265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc0c0047c0 msgr2=0x7fbc0c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:52.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.815+0000 7fbc17fff640 1 --2- 172.21.15.5:0/2516183265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc0c0047c0 0x7fbc0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:52.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.815+0000 7fbc1f7fe640 1 --2- 172.21.15.5:0/2516183265 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc0c0047c0 0x7fbc0c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.815+0000 7fbc1f7fe640 1 -- 172.21.15.5:0/2516183265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c003520 con 0x7fbc0c003970 2023-11-24T06:15:52.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.815+0000 7fbc1ffff640 1 -- 172.21.15.5:0/2516183265 reap_dead start 2023-11-24T06:15:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.816+0000 7fbc1f7fe640 1 --2- 172.21.15.5:0/2516183265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc0c003970 0x7fbc0c003d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000c240 tx=0x7fbc0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff78526c6b2d816 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.816+0000 7fbc1effd640 1 -- 172.21.15.5:0/2516183265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10005ad0 con 0x7fbc0c003970 2023-11-24T06:15:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.816+0000 7fbc1effd640 1 -- 172.21.15.5:0/2516183265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc10005c50 con 0x7fbc0c003970 2023-11-24T06:15:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.816+0000 7fbc1effd640 1 -- 172.21.15.5:0/2516183265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1002cab0 con 0x7fbc0c003970 2023-11-24T06:15:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.816+0000 7fbc25d01640 1 -- 172.21.15.5:0/2516183265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc0c003970 msgr2=0x7fbc0c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.816+0000 7fbc25d01640 1 --2- 172.21.15.5:0/2516183265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc0c003970 0x7fbc0c003d90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000c240 tx=0x7fbc0c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.817+0000 7fbc25d01640 1 -- 172.21.15.5:0/2516183265 shutdown_connections 2023-11-24T06:15:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.817+0000 7fbc25d01640 1 --2- 172.21.15.5:0/2516183265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc0c003970 0x7fbc0c003d90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.817+0000 7fbc25d01640 1 -- 172.21.15.5:0/2516183265 >> 172.21.15.5:0/2516183265 conn(0x7fbc200663d0 msgr2=0x7fbc200667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.817+0000 7fbc25d01640 1 -- 172.21.15.5:0/2516183265 shutdown_connections 2023-11-24T06:15:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.817+0000 7fbc25d01640 1 -- 172.21.15.5:0/2516183265 wait complete. 2023-11-24T06:15:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.818+0000 7fbc25d01640 1 Processor -- start 2023-11-24T06:15:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.819+0000 7fbc25d01640 1 -- start start 2023-11-24T06:15:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.819+0000 7fbc25d01640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc0c003970 0x7fbc201f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc25d01640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2015b180 0x7fbc2015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc25d01640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20158990 0x7fbc20158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc25d01640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc200cc610 con 0x7fbc20158990 2023-11-24T06:15:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc25d01640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2015bb20 con 0x7fbc2015b180 2023-11-24T06:15:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc25d01640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc201f67f0 con 0x7fbc0c003970 2023-11-24T06:15:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20158990 0x7fbc20158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20158990 0x7fbc20158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57578/0 (socket says 172.21.15.5:57578) 2023-11-24T06:15:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc1ffff640 1 -- 172.21.15.5:0/3131118726 learned_addr learned my addr 172.21.15.5:0/3131118726 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc1f7fe640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc0c003970 0x7fbc201f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc17fff640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2015b180 0x7fbc2015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc1ffff640 1 -- 172.21.15.5:0/3131118726 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc0c003970 msgr2=0x7fbc201f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc1ffff640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc0c003970 0x7fbc201f62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc1ffff640 1 -- 172.21.15.5:0/3131118726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2015b180 msgr2=0x7fbc2015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc1ffff640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2015b180 0x7fbc2015b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc1ffff640 1 -- 172.21.15.5:0/3131118726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc20159720 con 0x7fbc20158990 2023-11-24T06:15:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.820+0000 7fbc17fff640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2015b180 0x7fbc2015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:15:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.821+0000 7fbc1ffff640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20158990 0x7fbc20158e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800eaa0 tx=0x7fbc1800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.821+0000 7fbc1cff9640 1 -- 172.21.15.5:0/3131118726 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18010040 con 0x7fbc20158990 2023-11-24T06:15:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.821+0000 7fbc1cff9640 1 -- 172.21.15.5:0/3131118726 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1800bbd0 con 0x7fbc20158990 2023-11-24T06:15:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.821+0000 7fbc1cff9640 1 -- 172.21.15.5:0/3131118726 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18014400 con 0x7fbc20158990 2023-11-24T06:15:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.821+0000 7fbc25d01640 1 -- 172.21.15.5:0/3131118726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc20159a00 con 0x7fbc20158990 2023-11-24T06:15:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.821+0000 7fbc25d01640 1 -- 172.21.15.5:0/3131118726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc20159fc0 con 0x7fbc20158990 2023-11-24T06:15:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.822+0000 7fbc25d01640 1 -- 172.21.15.5:0/3131118726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc200cc780 con 0x7fbc20158990 2023-11-24T06:15:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.823+0000 7fbc1cff9640 1 -- 172.21.15.5:0/3131118726 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc18014560 con 0x7fbc20158990 2023-11-24T06:15:52.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.824+0000 7fbc1cff9640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbc0006ae70 0x7fbc0006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:52.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.824+0000 7fbc1cff9640 1 -- 172.21.15.5:0/3131118726 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc180898e0 con 0x7fbc20158990 2023-11-24T06:15:52.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.824+0000 7fbc1f7fe640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbc0006ae70 0x7fbc0006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:52.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.825+0000 7fbc1f7fe640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbc0006ae70 0x7fbc0006d330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc10034000 tx=0x7fbc10009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:52.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:52.828+0000 7fbc1cff9640 1 -- 172.21.15.5:0/3131118726 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc200cc780 con 0x7fbc20158990 2023-11-24T06:15:53.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.022+0000 7fbc25d01640 1 -- 172.21.15.5:0/3131118726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc2015a180 con 0x7fbc20158990 2023-11-24T06:15:53.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.023+0000 7fbc1cff9640 1 -- 172.21.15.5:0/3131118726 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16361 (secure 0 0 0) 0x7fbc180548f0 con 0x7fbc20158990 2023-11-24T06:15:53.027 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:15:53.027 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:52.779059+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:15:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.026+0000 7fbc25d01640 1 -- 172.21.15.5:0/3131118726 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbc0006ae70 msgr2=0x7fbc0006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.026+0000 7fbc25d01640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbc0006ae70 0x7fbc0006d330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbc10034000 tx=0x7fbc10009030 comp rx=0 tx=0).stop 2023-11-24T06:15:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.026+0000 7fbc25d01640 1 -- 172.21.15.5:0/3131118726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20158990 msgr2=0x7fbc20158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.026+0000 7fbc25d01640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20158990 0x7fbc20158e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbc1800eaa0 tx=0x7fbc1800ef60 comp rx=0 tx=0).stop 2023-11-24T06:15:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.026+0000 7fbc25d01640 1 -- 172.21.15.5:0/3131118726 shutdown_connections 2023-11-24T06:15:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.026+0000 7fbc25d01640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbc0006ae70 0x7fbc0006d330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.026+0000 7fbc25d01640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc20158990 0x7fbc20158e10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.026+0000 7fbc25d01640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbc2015b180 0x7fbc2015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.026+0000 7fbc25d01640 1 --2- 172.21.15.5:0/3131118726 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbc0c003970 0x7fbc201f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.026+0000 7fbc25d01640 1 -- 172.21.15.5:0/3131118726 >> 172.21.15.5:0/3131118726 conn(0x7fbc200663d0 msgr2=0x7fbc20143310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:53.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.027+0000 7fbc25d01640 1 -- 172.21.15.5:0/3131118726 shutdown_connections 2023-11-24T06:15:53.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.027+0000 7fbc25d01640 1 -- 172.21.15.5:0/3131118726 wait complete. 2023-11-24T06:15:53.050 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:15:53.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.280+0000 7efeacddc640 1 Processor -- start 2023-11-24T06:15:53.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.280+0000 7efeacddc640 1 -- start start 2023-11-24T06:15:53.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.281+0000 7efeacddc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efea8162600 0x7efea8162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.281+0000 7efeacddc640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7efea806c3c0 0x7efea806c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.281+0000 7efeacddc640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea8162fa0 con 0x7efea8162220 2023-11-24T06:15:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.281+0000 7efeacddc640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea806cd90 con 0x7efea806c3c0 2023-11-24T06:15:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.281+0000 7efeacddc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea80706a0 con 0x7efea8162600 2023-11-24T06:15:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.282+0000 7efea659c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efea8162220 0x7efea80b1ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56444/0 (socket says 172.21.15.5:56444) 2023-11-24T06:15:53.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.282+0000 7efea659c640 1 -- 172.21.15.5:0/2762649283 learned_addr learned my addr 172.21.15.5:0/2762649283 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:53.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.282+0000 7efea6d9d640 1 --2- 172.21.15.5:0/2762649283 >> v2:172.21.15.6:3300/0 conn(0x7efea806c3c0 0x7efea806c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:53.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.282+0000 7efea5d9b640 1 --2- 172.21.15.5:0/2762649283 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efea8162600 0x7efea8162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:53.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.282+0000 7efea559a640 1 -- 172.21.15.5:0/2762649283 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 682968101 0 0) 0x7efea8162fa0 con 0x7efea8162220 2023-11-24T06:15:53.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.282+0000 7efea559a640 1 -- 172.21.15.5:0/2762649283 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe94003520 con 0x7efea8162220 2023-11-24T06:15:53.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.282+0000 7efea559a640 1 -- 172.21.15.5:0/2762649283 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3218490392 0 0) 0x7efe94003520 con 0x7efea8162220 2023-11-24T06:15:53.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.282+0000 7efea559a640 1 -- 172.21.15.5:0/2762649283 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efea8162600 msgr2=0x7efea8162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:53.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.282+0000 7efea559a640 1 --2- 172.21.15.5:0/2762649283 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efea8162600 0x7efea8162a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:53.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea559a640 1 -- 172.21.15.5:0/2762649283 >> v2:172.21.15.6:3300/0 conn(0x7efea806c3c0 msgr2=0x7efea806c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:53.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea559a640 1 --2- 172.21.15.5:0/2762649283 >> v2:172.21.15.6:3300/0 conn(0x7efea806c3c0 0x7efea806c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:53.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea559a640 1 -- 172.21.15.5:0/2762649283 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea8070c60 con 0x7efea8162220 2023-11-24T06:15:53.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea559a640 1 -- 172.21.15.5:0/2762649283 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7efe980019f0 con 0x7efea8162220 2023-11-24T06:15:53.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea559a640 1 -- 172.21.15.5:0/2762649283 >> v1:172.21.15.5:6789/0 conn(0x7efea8162220 legacy=0x7efea80b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:53.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea559a640 1 --2- 172.21.15.5:0/2762649283 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efea8162600 0x7efe94003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:53.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea559a640 1 --2- 172.21.15.5:0/2762649283 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe94003f40 0x7efe940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:53.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea559a640 1 --2- 172.21.15.5:0/2762649283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe94004d00 0x7efe94005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:53.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea559a640 1 -- 172.21.15.5:0/2762649283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea806cd90 con 0x7efe94004d00 2023-11-24T06:15:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea559a640 1 -- 172.21.15.5:0/2762649283 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea80706a0 con 0x7efe94003f40 2023-11-24T06:15:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea559a640 1 -- 172.21.15.5:0/2762649283 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9400b980 con 0x7efea8162600 2023-11-24T06:15:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea6d9d640 1 --2- 172.21.15.5:0/2762649283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe94004d00 0x7efe94005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea659c640 1 --2- 172.21.15.5:0/2762649283 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efea8162600 0x7efe94003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:53.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea5d9b640 1 --2- 172.21.15.5:0/2762649283 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe94003f40 0x7efe940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea6d9d640 1 -- 172.21.15.5:0/2762649283 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efea8162600 msgr2=0x7efe94003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea6d9d640 1 --2- 172.21.15.5:0/2762649283 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efea8162600 0x7efe94003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea6d9d640 1 -- 172.21.15.5:0/2762649283 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe94003f40 msgr2=0x7efe940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea6d9d640 1 --2- 172.21.15.5:0/2762649283 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe94003f40 0x7efe940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.283+0000 7efea6d9d640 1 -- 172.21.15.5:0/2762649283 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe94003520 con 0x7efe94004d00 2023-11-24T06:15:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.285+0000 7efea6d9d640 1 --2- 172.21.15.5:0/2762649283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe94004d00 0x7efe94005180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efea0009af0 tx=0x7efe940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e42ae475eaaf07 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:53.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.285+0000 7efea559a640 1 -- 172.21.15.5:0/2762649283 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea002c710 con 0x7efe94004d00 2023-11-24T06:15:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.285+0000 7efea559a640 1 -- 172.21.15.5:0/2762649283 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea002c870 con 0x7efe94004d00 2023-11-24T06:15:53.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.285+0000 7efea559a640 1 -- 172.21.15.5:0/2762649283 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea002cc00 con 0x7efe94004d00 2023-11-24T06:15:53.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.285+0000 7efeacddc640 1 -- 172.21.15.5:0/2762649283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe94004d00 msgr2=0x7efe94005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:53.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.285+0000 7efeacddc640 1 --2- 172.21.15.5:0/2762649283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe94004d00 0x7efe94005180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efea0009af0 tx=0x7efe940057f0 comp rx=0 tx=0).stop 2023-11-24T06:15:53.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.286+0000 7efea6d9d640 1 -- 172.21.15.5:0/2762649283 reap_dead start 2023-11-24T06:15:53.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.286+0000 7efeacddc640 1 -- 172.21.15.5:0/2762649283 shutdown_connections 2023-11-24T06:15:53.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.286+0000 7efeacddc640 1 -- 172.21.15.5:0/2762649283 >> 172.21.15.5:0/2762649283 conn(0x7efea80b0ba0 msgr2=0x7efea80b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:53.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.286+0000 7efeacddc640 1 -- 172.21.15.5:0/2762649283 shutdown_connections 2023-11-24T06:15:53.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.286+0000 7efeacddc640 1 -- 172.21.15.5:0/2762649283 wait complete. 2023-11-24T06:15:53.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.288+0000 7efeacddc640 1 Processor -- start 2023-11-24T06:15:53.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.288+0000 7efeacddc640 1 -- start start 2023-11-24T06:15:53.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.289+0000 7efeacddc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe94004d00 0x7efea8164270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:53.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.289+0000 7efeacddc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea80b2ae0 0x7efea80b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:53.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.289+0000 7efeacddc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efea80ac320 0x7efea80ac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:53.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.289+0000 7efeacddc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea8163740 con 0x7efea80b2ae0 2023-11-24T06:15:53.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.289+0000 7efeacddc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea80b34a0 con 0x7efe94004d00 2023-11-24T06:15:53.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.289+0000 7efeacddc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea8165c20 con 0x7efea80ac320 2023-11-24T06:15:53.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.289+0000 7efea5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea80b2ae0 0x7efea80b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:53.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.290+0000 7efea5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea80b2ae0 0x7efea80b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35554/0 (socket says 172.21.15.5:35554) 2023-11-24T06:15:53.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.290+0000 7efea5d9b640 1 -- 172.21.15.5:0/3372990819 learned_addr learned my addr 172.21.15.5:0/3372990819 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:53.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.290+0000 7efea659c640 1 --2- 172.21.15.5:0/3372990819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe94004d00 0x7efea8164270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:53.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.290+0000 7efea6d9d640 1 --2- 172.21.15.5:0/3372990819 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efea80ac320 0x7efea80ac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:53.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.290+0000 7efea5d9b640 1 -- 172.21.15.5:0/3372990819 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efea80ac320 msgr2=0x7efea80ac7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:53.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.290+0000 7efea5d9b640 1 --2- 172.21.15.5:0/3372990819 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efea80ac320 0x7efea80ac7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:53.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.290+0000 7efea5d9b640 1 -- 172.21.15.5:0/3372990819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe94004d00 msgr2=0x7efea8164270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:53.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.290+0000 7efea5d9b640 1 --2- 172.21.15.5:0/3372990819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe94004d00 0x7efea8164270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:53.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.290+0000 7efea5d9b640 1 -- 172.21.15.5:0/3372990819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea8164b80 con 0x7efea80b2ae0 2023-11-24T06:15:53.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.290+0000 7efea5d9b640 1 --2- 172.21.15.5:0/3372990819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea80b2ae0 0x7efea80b2f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe90011540 tx=0x7efe90011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:53.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.291+0000 7efe9effd640 1 -- 172.21.15.5:0/3372990819 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90015930 con 0x7efea80b2ae0 2023-11-24T06:15:53.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.291+0000 7efe9effd640 1 -- 172.21.15.5:0/3372990819 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe90015a90 con 0x7efea80b2ae0 2023-11-24T06:15:53.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.291+0000 7efeacddc640 1 -- 172.21.15.5:0/3372990819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea8164e60 con 0x7efea80b2ae0 2023-11-24T06:15:53.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.291+0000 7efe9effd640 1 -- 172.21.15.5:0/3372990819 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90015e20 con 0x7efea80b2ae0 2023-11-24T06:15:53.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.291+0000 7efeacddc640 1 -- 172.21.15.5:0/3372990819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea8165420 con 0x7efea80b2ae0 2023-11-24T06:15:53.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.292+0000 7efeacddc640 1 -- 172.21.15.5:0/3372990819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea809d200 con 0x7efea80b2ae0 2023-11-24T06:15:53.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.293+0000 7efe9effd640 1 -- 172.21.15.5:0/3372990819 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efe9000f460 con 0x7efea80b2ae0 2023-11-24T06:15:53.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.294+0000 7efe9effd640 1 --2- 172.21.15.5:0/3372990819 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efe8806ae70 0x7efe8806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:53.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.295+0000 7efe9effd640 1 -- 172.21.15.5:0/3372990819 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe90089e50 con 0x7efea80b2ae0 2023-11-24T06:15:53.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.295+0000 7efea659c640 1 --2- 172.21.15.5:0/3372990819 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efe8806ae70 0x7efe8806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:53.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.295+0000 7efea659c640 1 --2- 172.21.15.5:0/3372990819 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efe8806ae70 0x7efe8806d330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efe980013d0 tx=0x7efe98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:53.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.299+0000 7efe9effd640 1 -- 172.21.15.5:0/3372990819 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efe90054e60 con 0x7efea80b2ae0 2023-11-24T06:15:53.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:53.531+0000 7efeacddc640 1 -- 172.21.15.5:0/3372990819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efea81660b0 con 0x7efea80b2ae0 2023-11-24T06:15:54.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.159+0000 7efe9effd640 1 -- 172.21.15.5:0/3372990819 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7efe90058480 con 0x7efea80b2ae0 2023-11-24T06:15:54.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.163+0000 7efeacddc640 1 -- 172.21.15.5:0/3372990819 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efe8806ae70 msgr2=0x7efe8806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:54.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.163+0000 7efeacddc640 1 --2- 172.21.15.5:0/3372990819 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efe8806ae70 0x7efe8806d330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efe980013d0 tx=0x7efe98009040 comp rx=0 tx=0).stop 2023-11-24T06:15:54.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.163+0000 7efeacddc640 1 -- 172.21.15.5:0/3372990819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea80b2ae0 msgr2=0x7efea80b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:54.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.163+0000 7efeacddc640 1 --2- 172.21.15.5:0/3372990819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea80b2ae0 0x7efea80b2f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe90011540 tx=0x7efe90011a00 comp rx=0 tx=0).stop 2023-11-24T06:15:54.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.163+0000 7efeacddc640 1 -- 172.21.15.5:0/3372990819 shutdown_connections 2023-11-24T06:15:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.163+0000 7efeacddc640 1 --2- 172.21.15.5:0/3372990819 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efe8806ae70 0x7efe8806d330 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.163+0000 7efeacddc640 1 --2- 172.21.15.5:0/3372990819 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efea80ac320 0x7efea80ac7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.163+0000 7efeacddc640 1 --2- 172.21.15.5:0/3372990819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efea80b2ae0 0x7efea80b2f60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.163+0000 7efeacddc640 1 --2- 172.21.15.5:0/3372990819 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe94004d00 0x7efea8164270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.163+0000 7efeacddc640 1 -- 172.21.15.5:0/3372990819 >> 172.21.15.5:0/3372990819 conn(0x7efea80b0ba0 msgr2=0x7efea80af450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.163+0000 7efeacddc640 1 -- 172.21.15.5:0/3372990819 shutdown_connections 2023-11-24T06:15:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.164+0000 7efeacddc640 1 -- 172.21.15.5:0/3372990819 wait complete. 2023-11-24T06:15:54.189 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:15:54.189 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:15:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.412+0000 7f819a306640 1 Processor -- start 2023-11-24T06:15:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.412+0000 7f819a306640 1 -- start start 2023-11-24T06:15:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.413+0000 7f819a306640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8194160ab0 0x7f81940d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.413+0000 7f819a306640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f81940ce6f0 0x7f81940cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.413+0000 7f819a306640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81940cfae0 0x7f81940d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.413+0000 7f819a306640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81940cff90 con 0x7f81940ce6f0 2023-11-24T06:15:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.413+0000 7f819a306640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81940cf190 con 0x7f8194160ab0 2023-11-24T06:15:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.413+0000 7f819a306640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81940b2e60 con 0x7f81940cfae0 2023-11-24T06:15:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.414+0000 7f8198b03640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f81940ce6f0 0x7f81940cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.414+0000 7f8198b03640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f81940ce6f0 0x7f81940cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35558/0 (socket says 172.21.15.5:35558) 2023-11-24T06:15:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.414+0000 7f8198b03640 1 -- 172.21.15.5:0/3315007087 learned_addr learned my addr 172.21.15.5:0/3315007087 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.414+0000 7f8198b03640 1 -- 172.21.15.5:0/3315007087 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81940cfae0 msgr2=0x7f81940d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.414+0000 7f8198b03640 1 --2- 172.21.15.5:0/3315007087 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81940cfae0 0x7f81940d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.414+0000 7f8198b03640 1 -- 172.21.15.5:0/3315007087 >> v2:172.21.15.6:3300/0 conn(0x7f8194160ab0 msgr2=0x7f81940d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.414+0000 7f8198b03640 1 --2- 172.21.15.5:0/3315007087 >> v2:172.21.15.6:3300/0 conn(0x7f8194160ab0 0x7f81940d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.414+0000 7f8198b03640 1 -- 172.21.15.5:0/3315007087 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81940cebd0 con 0x7f81940ce6f0 2023-11-24T06:15:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.415+0000 7f8198b03640 1 --2- 172.21.15.5:0/3315007087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81940ce6f0 0x7f81940cf5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8194068030 tx=0x7f818802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c452eba92eb9ee5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.416+0000 7f8193fff640 1 -- 172.21.15.5:0/3315007087 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81880317e0 con 0x7f81940ce6f0 2023-11-24T06:15:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.416+0000 7f8193fff640 1 -- 172.21.15.5:0/3315007087 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8188031940 con 0x7f81940ce6f0 2023-11-24T06:15:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.416+0000 7f8193fff640 1 -- 172.21.15.5:0/3315007087 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188031cd0 con 0x7f81940ce6f0 2023-11-24T06:15:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.416+0000 7f819a306640 1 -- 172.21.15.5:0/3315007087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81940ce6f0 msgr2=0x7f81940cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.416+0000 7f819a306640 1 --2- 172.21.15.5:0/3315007087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81940ce6f0 0x7f81940cf5a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8194068030 tx=0x7f818802b880 comp rx=0 tx=0).stop 2023-11-24T06:15:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.417+0000 7f819a306640 1 -- 172.21.15.5:0/3315007087 shutdown_connections 2023-11-24T06:15:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.417+0000 7f819a306640 1 --2- 172.21.15.5:0/3315007087 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81940cfae0 0x7f81940d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.417+0000 7f819a306640 1 --2- 172.21.15.5:0/3315007087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81940ce6f0 0x7f81940cf5a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.417+0000 7f819a306640 1 --2- 172.21.15.5:0/3315007087 >> v2:172.21.15.6:3300/0 conn(0x7f8194160ab0 0x7f81940d3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.417+0000 7f819a306640 1 -- 172.21.15.5:0/3315007087 >> 172.21.15.5:0/3315007087 conn(0x7f81940d3060 msgr2=0x7f81940d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.417+0000 7f819a306640 1 -- 172.21.15.5:0/3315007087 shutdown_connections 2023-11-24T06:15:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.417+0000 7f819a306640 1 -- 172.21.15.5:0/3315007087 wait complete. 2023-11-24T06:15:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.419+0000 7f819a306640 1 Processor -- start 2023-11-24T06:15:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.419+0000 7f819a306640 1 -- start start 2023-11-24T06:15:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.420+0000 7f819a306640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81940ce6f0 0x7f8194159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.420+0000 7f819a306640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81940cfae0 0x7f819415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.420+0000 7f819a306640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8194160ab0 0x7f81940c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.420+0000 7f819a306640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819415a8d0 con 0x7f8194160ab0 2023-11-24T06:15:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.420+0000 7f819a306640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819415abb0 con 0x7f81940ce6f0 2023-11-24T06:15:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.420+0000 7f819a306640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819415ae90 con 0x7f81940cfae0 2023-11-24T06:15:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.420+0000 7f8199b05640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8194160ab0 0x7f81940c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.420+0000 7f8199b05640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8194160ab0 0x7f81940c6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35562/0 (socket says 172.21.15.5:35562) 2023-11-24T06:15:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.420+0000 7f8199b05640 1 -- 172.21.15.5:0/1588697670 learned_addr learned my addr 172.21.15.5:0/1588697670 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.420+0000 7f8198b03640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81940cfae0 0x7f819415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.420+0000 7f8199304640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81940ce6f0 0x7f8194159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.421+0000 7f8199b05640 1 -- 172.21.15.5:0/1588697670 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81940cfae0 msgr2=0x7f819415a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.421+0000 7f8199b05640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81940cfae0 0x7f819415a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.421+0000 7f8199b05640 1 -- 172.21.15.5:0/1588697670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81940ce6f0 msgr2=0x7f8194159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.421+0000 7f8199b05640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81940ce6f0 0x7f8194159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.421+0000 7f8199b05640 1 -- 172.21.15.5:0/1588697670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81880094d0 con 0x7f8194160ab0 2023-11-24T06:15:54.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.421+0000 7f8198b03640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81940cfae0 0x7f819415a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:15:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.421+0000 7f8199b05640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8194160ab0 0x7f81940c6fd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f818c009fd0 tx=0x7f818c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.421+0000 7f8191ffb640 1 -- 172.21.15.5:0/1588697670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c011830 con 0x7f8194160ab0 2023-11-24T06:15:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.421+0000 7f8191ffb640 1 -- 172.21.15.5:0/1588697670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818c011990 con 0x7f8194160ab0 2023-11-24T06:15:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.421+0000 7f8191ffb640 1 -- 172.21.15.5:0/1588697670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c011d20 con 0x7f8194160ab0 2023-11-24T06:15:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.421+0000 7f819a306640 1 -- 172.21.15.5:0/1588697670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819415b170 con 0x7f8194160ab0 2023-11-24T06:15:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.421+0000 7f819a306640 1 -- 172.21.15.5:0/1588697670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819415a5a0 con 0x7f8194160ab0 2023-11-24T06:15:54.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.423+0000 7f819a306640 1 -- 172.21.15.5:0/1588697670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81940ca4c0 con 0x7f8194160ab0 2023-11-24T06:15:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.423+0000 7f8191ffb640 1 -- 172.21.15.5:0/1588697670 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f818c01b460 con 0x7f8194160ab0 2023-11-24T06:15:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.424+0000 7f8191ffb640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f818006ae70 0x7f818006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.424+0000 7f8191ffb640 1 -- 172.21.15.5:0/1588697670 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f818c089480 con 0x7f8194160ab0 2023-11-24T06:15:54.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.425+0000 7f8199304640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f818006ae70 0x7f818006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:54.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.425+0000 7f8199304640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f818006ae70 0x7f818006d330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8184007dc0 tx=0x7f81840072f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.429+0000 7f8191ffb640 1 -- 172.21.15.5:0/1588697670 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f81940ca4c0 con 0x7f8194160ab0 2023-11-24T06:15:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:54.662+0000 7f819a306640 1 -- 172.21.15.5:0/1588697670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f81940ca4c0 con 0x7f8194160ab0 2023-11-24T06:15:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.163+0000 7f8191ffb640 1 -- 172.21.15.5:0/1588697670 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f81940ca4c0 con 0x7f8194160ab0 2023-11-24T06:15:55.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.167+0000 7f819a306640 1 -- 172.21.15.5:0/1588697670 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f818006ae70 msgr2=0x7f818006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.167+0000 7f819a306640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f818006ae70 0x7f818006d330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8184007dc0 tx=0x7f81840072f0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.167+0000 7f819a306640 1 -- 172.21.15.5:0/1588697670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8194160ab0 msgr2=0x7f81940c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.167+0000 7f819a306640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8194160ab0 0x7f81940c6fd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f818c009fd0 tx=0x7f818c00b8e0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.167+0000 7f819a306640 1 -- 172.21.15.5:0/1588697670 shutdown_connections 2023-11-24T06:15:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.167+0000 7f819a306640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f818006ae70 0x7f818006d330 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.167+0000 7f819a306640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81940cfae0 0x7f819415a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.167+0000 7f819a306640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8194160ab0 0x7f81940c6fd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.167+0000 7f819a306640 1 --2- 172.21.15.5:0/1588697670 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81940ce6f0 0x7f8194159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.167+0000 7f819a306640 1 -- 172.21.15.5:0/1588697670 >> 172.21.15.5:0/1588697670 conn(0x7f81940d3060 msgr2=0x7f819406c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.168+0000 7f819a306640 1 -- 172.21.15.5:0/1588697670 shutdown_connections 2023-11-24T06:15:55.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.168+0000 7f819a306640 1 -- 172.21.15.5:0/1588697670 wait complete. 2023-11-24T06:15:55.194 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:15:55.194 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:15:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.417+0000 7f8c0d05c640 1 Processor -- start 2023-11-24T06:15:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.417+0000 7f8c0d05c640 1 -- start start 2023-11-24T06:15:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.418+0000 7f8c0d05c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c08161930 0x7f8c080ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.419+0000 7f8c0d05c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8c0806d520 0x7f8c0806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.419+0000 7f8c0d05c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c080ad510 con 0x7f8c080719c0 2023-11-24T06:15:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.419+0000 7f8c0d05c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c080acc50 con 0x7f8c08161930 2023-11-24T06:15:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.419+0000 7f8c0d05c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c0806de70 con 0x7f8c0806d520 2023-11-24T06:15:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.419+0000 7f8c06d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c08161930 0x7f8c080ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.419+0000 7f8c06d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c08161930 0x7f8c080ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35570/0 (socket says 172.21.15.5:35570) 2023-11-24T06:15:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.419+0000 7f8c06d9d640 1 -- 172.21.15.5:0/1920109206 learned_addr learned my addr 172.21.15.5:0/1920109206 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.419+0000 7f8c0759e640 1 --2- 172.21.15.5:0/1920109206 >> v2:172.21.15.6:3300/0 conn(0x7f8c0806d520 0x7f8c0806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.419+0000 7f8c06d9d640 1 -- 172.21.15.5:0/1920109206 >> v2:172.21.15.6:3300/0 conn(0x7f8c0806d520 msgr2=0x7f8c0806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.419+0000 7f8c06d9d640 1 --2- 172.21.15.5:0/1920109206 >> v2:172.21.15.6:3300/0 conn(0x7f8c0806d520 0x7f8c0806d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.419+0000 7f8c06d9d640 1 -- 172.21.15.5:0/1920109206 >> v1:172.21.15.6:6789/0 conn(0x7f8c080719c0 legacy=0x7f8c080ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.419+0000 7f8c06d9d640 1 -- 172.21.15.5:0/1920109206 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c08071e60 con 0x7f8c08161930 2023-11-24T06:15:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.420+0000 7f8c06d9d640 1 --2- 172.21.15.5:0/1920109206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c08161930 0x7f8c080ae6a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8c08070440 tx=0x7f8bf802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebd6848af1f9c6a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.421+0000 7f8c05d9b640 1 -- 172.21.15.5:0/1920109206 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8031870 con 0x7f8c08161930 2023-11-24T06:15:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.421+0000 7f8c05d9b640 1 -- 172.21.15.5:0/1920109206 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf80319d0 con 0x7f8c08161930 2023-11-24T06:15:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.421+0000 7f8c05d9b640 1 -- 172.21.15.5:0/1920109206 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8031d60 con 0x7f8c08161930 2023-11-24T06:15:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.422+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1920109206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c08161930 msgr2=0x7f8c080ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.422+0000 7f8c0d05c640 1 --2- 172.21.15.5:0/1920109206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c08161930 0x7f8c080ae6a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8c08070440 tx=0x7f8bf802b910 comp rx=0 tx=0).stop 2023-11-24T06:15:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.422+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1920109206 shutdown_connections 2023-11-24T06:15:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.422+0000 7f8c0d05c640 1 --2- 172.21.15.5:0/1920109206 >> v2:172.21.15.6:3300/0 conn(0x7f8c0806d520 0x7f8c0806d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.422+0000 7f8c0d05c640 1 --2- 172.21.15.5:0/1920109206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c08161930 0x7f8c080ae6a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.422+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1920109206 >> 172.21.15.5:0/1920109206 conn(0x7f8c080b0d80 msgr2=0x7f8c080b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.422+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1920109206 shutdown_connections 2023-11-24T06:15:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.422+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1920109206 wait complete. 2023-11-24T06:15:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.424+0000 7f8c0d05c640 1 Processor -- start 2023-11-24T06:15:55.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.424+0000 7f8c0d05c640 1 -- start start 2023-11-24T06:15:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.425+0000 7f8c0d05c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c0806d520 0x7f8c0815f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.425+0000 7f8c0d05c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c080719c0 0x7f8c0815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:55.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.425+0000 7f8c0d05c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c08161930 0x7f8c0815b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.425+0000 7f8c0d05c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0815ba40 con 0x7f8c08161930 2023-11-24T06:15:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.425+0000 7f8c0d05c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08099b30 con 0x7f8c0806d520 2023-11-24T06:15:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.425+0000 7f8c0d05c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080cc770 con 0x7f8c080719c0 2023-11-24T06:15:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.425+0000 7f8c0759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c08161930 0x7f8c0815b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.425+0000 7f8c0759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c08161930 0x7f8c0815b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35580/0 (socket says 172.21.15.5:35580) 2023-11-24T06:15:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.425+0000 7f8c0759e640 1 -- 172.21.15.5:0/1965376163 learned_addr learned my addr 172.21.15.5:0/1965376163 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.425+0000 7f8c06d9d640 1 --2- 172.21.15.5:0/1965376163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c0806d520 0x7f8c0815f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.425+0000 7f8c0659c640 1 --2- 172.21.15.5:0/1965376163 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c080719c0 0x7f8c0815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.426+0000 7f8c0759e640 1 -- 172.21.15.5:0/1965376163 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c080719c0 msgr2=0x7f8c0815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.426+0000 7f8c0759e640 1 --2- 172.21.15.5:0/1965376163 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c080719c0 0x7f8c0815afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.426+0000 7f8c0759e640 1 -- 172.21.15.5:0/1965376163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c0806d520 msgr2=0x7f8c0815f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.426+0000 7f8c0759e640 1 --2- 172.21.15.5:0/1965376163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c0806d520 0x7f8c0815f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.426+0000 7f8c0759e640 1 -- 172.21.15.5:0/1965376163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf80094d0 con 0x7f8c08161930 2023-11-24T06:15:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.426+0000 7f8c0759e640 1 --2- 172.21.15.5:0/1965376163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c08161930 0x7f8c0815b500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8c00007e40 tx=0x7f8c0000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.426+0000 7f8bef7fe640 1 -- 172.21.15.5:0/1965376163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00014620 con 0x7f8c08161930 2023-11-24T06:15:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.426+0000 7f8bef7fe640 1 -- 172.21.15.5:0/1965376163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0000ee90 con 0x7f8c08161930 2023-11-24T06:15:55.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.426+0000 7f8bef7fe640 1 -- 172.21.15.5:0/1965376163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c000149e0 con 0x7f8c08161930 2023-11-24T06:15:55.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.426+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1965376163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c080cca50 con 0x7f8c08161930 2023-11-24T06:15:55.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.426+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1965376163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c080cd010 con 0x7f8c08161930 2023-11-24T06:15:55.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.428+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1965376163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c080cd1d0 con 0x7f8c08161930 2023-11-24T06:15:55.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.429+0000 7f8bef7fe640 1 -- 172.21.15.5:0/1965376163 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c00014b40 con 0x7f8c08161930 2023-11-24T06:15:55.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.429+0000 7f8bef7fe640 1 --2- 172.21.15.5:0/1965376163 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8bf406ae40 0x7f8bf406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:55.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.430+0000 7f8bef7fe640 1 -- 172.21.15.5:0/1965376163 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c0008a060 con 0x7f8c08161930 2023-11-24T06:15:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.430+0000 7f8c06d9d640 1 --2- 172.21.15.5:0/1965376163 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8bf406ae40 0x7f8bf406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:55.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.430+0000 7f8c06d9d640 1 --2- 172.21.15.5:0/1965376163 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8bf406ae40 0x7f8bf406d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8036000 tx=0x7f8bf803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:55.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.434+0000 7f8bef7fe640 1 -- 172.21.15.5:0/1965376163 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c000bce90 con 0x7f8c08161930 2023-11-24T06:15:55.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.668+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1965376163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c08074770 con 0x7f8c08161930 2023-11-24T06:15:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.669+0000 7f8bef7fe640 1 -- 172.21.15.5:0/1965376163 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4895 (secure 0 0 0) 0x7f8c00055070 con 0x7f8c08161930 2023-11-24T06:15:55.672 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:15:55.672 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4009460946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4009460946},{"type":"v1","addr":"172.21.15.5:6837","nonce":4009460946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2496926427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2496926427},{"type":"v1","addr":"172.21.15.5:6836","nonce":2496926427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3829754769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3829754769},{"type":"v1","addr":"172.21.15.6:6835","nonce":3829754769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:15:51.611541+0000","modified":"2023-11-24T06:15:54.664234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6833/2760317954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2760317954},{"type":"v1","addr":"172.21.15.6:6833","nonce":2760317954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:15:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2023-11-24T06:15:55.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.671+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1965376163 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8bf406ae40 msgr2=0x7f8bf406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:55.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.671+0000 7f8c0d05c640 1 --2- 172.21.15.5:0/1965376163 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8bf406ae40 0x7f8bf406d300 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8036000 tx=0x7f8bf803c040 comp rx=0 tx=0).stop 2023-11-24T06:15:55.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.671+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1965376163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c08161930 msgr2=0x7f8c0815b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:55.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.671+0000 7f8c0d05c640 1 --2- 172.21.15.5:0/1965376163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c08161930 0x7f8c0815b500 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8c00007e40 tx=0x7f8c0000e4b0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.672+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1965376163 shutdown_connections 2023-11-24T06:15:55.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.672+0000 7f8c0d05c640 1 --2- 172.21.15.5:0/1965376163 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8bf406ae40 0x7f8bf406d300 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.672+0000 7f8c0d05c640 1 --2- 172.21.15.5:0/1965376163 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c080719c0 0x7f8c0815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.672+0000 7f8c0d05c640 1 --2- 172.21.15.5:0/1965376163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c08161930 0x7f8c0815b500 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.672+0000 7f8c0d05c640 1 --2- 172.21.15.5:0/1965376163 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c0806d520 0x7f8c0815f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.672+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1965376163 >> 172.21.15.5:0/1965376163 conn(0x7f8c080b0d80 msgr2=0x7f8c080ae950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:55.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.672+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1965376163 shutdown_connections 2023-11-24T06:15:55.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.672+0000 7f8c0d05c640 1 -- 172.21.15.5:0/1965376163 wait complete. 2023-11-24T06:15:55.695 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:15:55.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.919+0000 7fd67b06d640 1 Processor -- start 2023-11-24T06:15:55.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.919+0000 7fd67b06d640 1 -- start start 2023-11-24T06:15:55.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.920+0000 7fd67b06d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6740cf050 0x7fd6740cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:55.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.920+0000 7fd67b06d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd6740cd7b0 0x7fd6740cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:55.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.920+0000 7fd67b06d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd67415fa70 con 0x7fd67413dfd0 2023-11-24T06:15:55.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.920+0000 7fd67b06d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67415c580 con 0x7fd6740cd7b0 2023-11-24T06:15:55.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.920+0000 7fd67b06d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740cfa10 con 0x7fd6740cf050 2023-11-24T06:15:55.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.921+0000 7fd67a86c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd6740cd7b0 0x7fd6740cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:55.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.921+0000 7fd67986a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6740cf050 0x7fd6740cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:55.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.921+0000 7fd67a86c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd6740cd7b0 0x7fd6740cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:49828/0 (socket says 172.21.15.5:49828) 2023-11-24T06:15:55.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.921+0000 7fd67a86c640 1 -- 172.21.15.5:0/1584918226 learned_addr learned my addr 172.21.15.5:0/1584918226 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:55.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.922+0000 7fd67986a640 1 -- 172.21.15.5:0/1584918226 >> v2:172.21.15.6:3300/0 conn(0x7fd6740cd7b0 msgr2=0x7fd6740cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:55.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.922+0000 7fd67986a640 1 --2- 172.21.15.5:0/1584918226 >> v2:172.21.15.6:3300/0 conn(0x7fd6740cd7b0 0x7fd6740cdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.922+0000 7fd67986a640 1 -- 172.21.15.5:0/1584918226 >> v1:172.21.15.6:6789/0 conn(0x7fd67413dfd0 legacy=0x7fd67415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:55.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.922+0000 7fd67986a640 1 -- 172.21.15.5:0/1584918226 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd674140900 con 0x7fd6740cf050 2023-11-24T06:15:55.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.923+0000 7fd67986a640 1 --2- 172.21.15.5:0/1584918226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6740cf050 0x7fd6740cf4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd6640097a0 tx=0x7fd66402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca1c3a60de0be2d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:55.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.924+0000 7fd679069640 1 -- 172.21.15.5:0/1584918226 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664031830 con 0x7fd6740cf050 2023-11-24T06:15:55.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.924+0000 7fd679069640 1 -- 172.21.15.5:0/1584918226 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd664031990 con 0x7fd6740cf050 2023-11-24T06:15:55.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.924+0000 7fd679069640 1 -- 172.21.15.5:0/1584918226 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664031d20 con 0x7fd6740cf050 2023-11-24T06:15:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.924+0000 7fd67b06d640 1 -- 172.21.15.5:0/1584918226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6740cf050 msgr2=0x7fd6740cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.924+0000 7fd67b06d640 1 --2- 172.21.15.5:0/1584918226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6740cf050 0x7fd6740cf4d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd6640097a0 tx=0x7fd66402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.924+0000 7fd67b06d640 1 -- 172.21.15.5:0/1584918226 shutdown_connections 2023-11-24T06:15:55.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.924+0000 7fd67b06d640 1 --2- 172.21.15.5:0/1584918226 >> v2:172.21.15.6:3300/0 conn(0x7fd6740cd7b0 0x7fd6740cdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.924+0000 7fd67b06d640 1 --2- 172.21.15.5:0/1584918226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6740cf050 0x7fd6740cf4d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.924+0000 7fd67b06d640 1 -- 172.21.15.5:0/1584918226 >> 172.21.15.5:0/1584918226 conn(0x7fd67406a860 msgr2=0x7fd67406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:55.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.924+0000 7fd67b06d640 1 -- 172.21.15.5:0/1584918226 shutdown_connections 2023-11-24T06:15:55.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.924+0000 7fd67b06d640 1 -- 172.21.15.5:0/1584918226 wait complete. 2023-11-24T06:15:55.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.926+0000 7fd67b06d640 1 Processor -- start 2023-11-24T06:15:55.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.926+0000 7fd67b06d640 1 -- start start 2023-11-24T06:15:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.927+0000 7fd67b06d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6740cd7b0 0x7fd674070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.927+0000 7fd67b06d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6740cf050 0x7fd67413a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.927+0000 7fd67b06d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd67413dfd0 0x7fd674154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.927+0000 7fd67b06d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67413a9f0 con 0x7fd6740cd7b0 2023-11-24T06:15:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.927+0000 7fd67b06d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd67413ac90 con 0x7fd67413dfd0 2023-11-24T06:15:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.927+0000 7fd67b06d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67413af70 con 0x7fd6740cf050 2023-11-24T06:15:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.928+0000 7fd67a06b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6740cd7b0 0x7fd674070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.928+0000 7fd67a06b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6740cd7b0 0x7fd674070c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35584/0 (socket says 172.21.15.5:35584) 2023-11-24T06:15:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.928+0000 7fd67a06b640 1 -- 172.21.15.5:0/2691778993 learned_addr learned my addr 172.21.15.5:0/2691778993 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.928+0000 7fd67a86c640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd67413dfd0 0x7fd674154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.928+0000 7fd67986a640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6740cf050 0x7fd67413a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.928+0000 7fd67a06b640 1 -- 172.21.15.5:0/2691778993 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6740cf050 msgr2=0x7fd67413a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.928+0000 7fd67a06b640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6740cf050 0x7fd67413a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.928+0000 7fd67a06b640 1 -- 172.21.15.5:0/2691778993 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd67413dfd0 msgr2=0x7fd674154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.928+0000 7fd67a06b640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd67413dfd0 0x7fd674154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.928+0000 7fd67a06b640 1 -- 172.21.15.5:0/2691778993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd664009400 con 0x7fd6740cd7b0 2023-11-24T06:15:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.928+0000 7fd67a86c640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd67413dfd0 0x7fd674154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:15:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.928+0000 7fd67a06b640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6740cd7b0 0x7fd674070c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd67000c3c0 tx=0x7fd67000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.929+0000 7fd66affd640 1 -- 172.21.15.5:0/2691778993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6700052d0 con 0x7fd6740cd7b0 2023-11-24T06:15:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.929+0000 7fd66affd640 1 -- 172.21.15.5:0/2691778993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd670005450 con 0x7fd6740cd7b0 2023-11-24T06:15:55.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.929+0000 7fd66affd640 1 -- 172.21.15.5:0/2691778993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd670007260 con 0x7fd6740cd7b0 2023-11-24T06:15:55.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.929+0000 7fd67b06d640 1 -- 172.21.15.5:0/2691778993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd67413b250 con 0x7fd6740cd7b0 2023-11-24T06:15:55.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.929+0000 7fd67b06d640 1 -- 172.21.15.5:0/2691778993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67413b810 con 0x7fd6740cd7b0 2023-11-24T06:15:55.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.930+0000 7fd67b06d640 1 -- 172.21.15.5:0/2691778993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd67404e840 con 0x7fd6740cd7b0 2023-11-24T06:15:55.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.931+0000 7fd66affd640 1 -- 172.21.15.5:0/2691778993 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd6700073c0 con 0x7fd6740cd7b0 2023-11-24T06:15:55.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.932+0000 7fd66affd640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd65806ae40 0x7fd65806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:55.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.932+0000 7fd66affd640 1 -- 172.21.15.5:0/2691778993 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd670088a50 con 0x7fd6740cd7b0 2023-11-24T06:15:55.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.932+0000 7fd67986a640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd65806ae40 0x7fd65806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:55.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.933+0000 7fd67986a640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd65806ae40 0x7fd65806d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd664036000 tx=0x7fd66403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:55.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:55.937+0000 7fd66affd640 1 -- 172.21.15.5:0/2691778993 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd670053a60 con 0x7fd6740cd7b0 2023-11-24T06:15:56.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.130+0000 7fd67b06d640 1 -- 172.21.15.5:0/2691778993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6740ce9a0 con 0x7fd6740cd7b0 2023-11-24T06:15:56.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.133+0000 7fd66affd640 1 -- 172.21.15.5:0/2691778993 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16361 (secure 0 0 0) 0x7fd670057080 con 0x7fd6740cd7b0 2023-11-24T06:15:56.136 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:15:56.136 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:52.779059+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:15:56.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.135+0000 7fd67b06d640 1 -- 172.21.15.5:0/2691778993 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd65806ae40 msgr2=0x7fd65806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.135+0000 7fd67b06d640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd65806ae40 0x7fd65806d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd664036000 tx=0x7fd66403c040 comp rx=0 tx=0).stop 2023-11-24T06:15:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.135+0000 7fd67b06d640 1 -- 172.21.15.5:0/2691778993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6740cd7b0 msgr2=0x7fd674070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.135+0000 7fd67b06d640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6740cd7b0 0x7fd674070c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd67000c3c0 tx=0x7fd67000c880 comp rx=0 tx=0).stop 2023-11-24T06:15:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.135+0000 7fd67b06d640 1 -- 172.21.15.5:0/2691778993 shutdown_connections 2023-11-24T06:15:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.135+0000 7fd67b06d640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd65806ae40 0x7fd65806d300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.135+0000 7fd67b06d640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd6740cf050 0x7fd67413a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.136+0000 7fd67b06d640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd67413dfd0 0x7fd674154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.136+0000 7fd67b06d640 1 --2- 172.21.15.5:0/2691778993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6740cd7b0 0x7fd674070c80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.136+0000 7fd67b06d640 1 -- 172.21.15.5:0/2691778993 >> 172.21.15.5:0/2691778993 conn(0x7fd67406a860 msgr2=0x7fd67413e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:56.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.136+0000 7fd67b06d640 1 -- 172.21.15.5:0/2691778993 shutdown_connections 2023-11-24T06:15:56.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.136+0000 7fd67b06d640 1 -- 172.21.15.5:0/2691778993 wait complete. 2023-11-24T06:15:56.155 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:15:56.156 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:15:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.342+0000 7fa3e7f89640 1 Processor -- start 2023-11-24T06:15:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.343+0000 7fa3e7f89640 1 -- start start 2023-11-24T06:15:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.344+0000 7fa3e7f89640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3e0160970 0x7fa3e00b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.344+0000 7fa3e7f89640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa3e00b3140 0x7fa3e006d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.344+0000 7fa3e7f89640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e0071cd0 con 0x7fa3e0070780 2023-11-24T06:15:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.344+0000 7fa3e7f89640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e00b2120 con 0x7fa3e00b3140 2023-11-24T06:15:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.344+0000 7fa3e7f89640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00b2400 con 0x7fa3e0160970 2023-11-24T06:15:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.344+0000 7fa3e5524640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa3e00b3140 0x7fa3e006d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.344+0000 7fa3e5524640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa3e00b3140 0x7fa3e006d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35598/0 (socket says 172.21.15.5:35598) 2023-11-24T06:15:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.344+0000 7fa3e5524640 1 -- 172.21.15.5:0/642512965 learned_addr learned my addr 172.21.15.5:0/642512965 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.345+0000 7fa3e5d25640 1 --2- 172.21.15.5:0/642512965 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3e0160970 0x7fa3e00b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.345+0000 7fa3e5524640 1 -- 172.21.15.5:0/642512965 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3e0160970 msgr2=0x7fa3e00b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.345+0000 7fa3e5524640 1 --2- 172.21.15.5:0/642512965 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3e0160970 0x7fa3e00b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.345+0000 7fa3e5524640 1 -- 172.21.15.5:0/642512965 >> v1:172.21.15.6:6789/0 conn(0x7fa3e0070780 legacy=0x7fa3e00acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.345+0000 7fa3e5524640 1 -- 172.21.15.5:0/642512965 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00b26e0 con 0x7fa3e00b3140 2023-11-24T06:15:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.345+0000 7fa3e5d25640 1 --2- 172.21.15.5:0/642512965 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3e0160970 0x7fa3e00b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:15:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.346+0000 7fa3e5524640 1 --2- 172.21.15.5:0/642512965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e00b3140 0x7fa3e006d520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009870 tx=0x7fa3d002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59286299f022b091 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.347+0000 7fa3e4d23640 1 -- 172.21.15.5:0/642512965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00317c0 con 0x7fa3e00b3140 2023-11-24T06:15:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.347+0000 7fa3e4d23640 1 -- 172.21.15.5:0/642512965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d0031920 con 0x7fa3e00b3140 2023-11-24T06:15:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.347+0000 7fa3e4d23640 1 -- 172.21.15.5:0/642512965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0031cb0 con 0x7fa3e00b3140 2023-11-24T06:15:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.347+0000 7fa3e7f89640 1 -- 172.21.15.5:0/642512965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e00b3140 msgr2=0x7fa3e006d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.347+0000 7fa3e7f89640 1 --2- 172.21.15.5:0/642512965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e00b3140 0x7fa3e006d520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009870 tx=0x7fa3d002b860 comp rx=0 tx=0).stop 2023-11-24T06:15:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.348+0000 7fa3e7f89640 1 -- 172.21.15.5:0/642512965 shutdown_connections 2023-11-24T06:15:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.348+0000 7fa3e7f89640 1 --2- 172.21.15.5:0/642512965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e00b3140 0x7fa3e006d520 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.348+0000 7fa3e7f89640 1 --2- 172.21.15.5:0/642512965 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3e0160970 0x7fa3e00b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.348+0000 7fa3e7f89640 1 -- 172.21.15.5:0/642512965 >> 172.21.15.5:0/642512965 conn(0x7fa3e00b0d80 msgr2=0x7fa3e00b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.348+0000 7fa3e7f89640 1 -- 172.21.15.5:0/642512965 shutdown_connections 2023-11-24T06:15:56.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.348+0000 7fa3e7f89640 1 -- 172.21.15.5:0/642512965 wait complete. 2023-11-24T06:15:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.350+0000 7fa3e7f89640 1 Processor -- start 2023-11-24T06:15:56.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.350+0000 7fa3e7f89640 1 -- start start 2023-11-24T06:15:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.351+0000 7fa3e7f89640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3e0070780 0x7fa3e009cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.351+0000 7fa3e7f89640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e00b3140 0x7fa3e00acb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e7f89640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3e0160970 0x7fa3e0152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e7f89640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00ad070 con 0x7fa3e00b3140 2023-11-24T06:15:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e7f89640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00ad2f0 con 0x7fa3e0070780 2023-11-24T06:15:56.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e7f89640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00ad5d0 con 0x7fa3e0160970 2023-11-24T06:15:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e5524640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e00b3140 0x7fa3e00acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e5524640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e00b3140 0x7fa3e00acb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35610/0 (socket says 172.21.15.5:35610) 2023-11-24T06:15:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e5524640 1 -- 172.21.15.5:0/3230056277 learned_addr learned my addr 172.21.15.5:0/3230056277 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e5d25640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3e0070780 0x7fa3e009cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e6526640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3e0160970 0x7fa3e0152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e5524640 1 -- 172.21.15.5:0/3230056277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3e0160970 msgr2=0x7fa3e0152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e5524640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3e0160970 0x7fa3e0152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e5524640 1 -- 172.21.15.5:0/3230056277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3e0070780 msgr2=0x7fa3e009cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e5524640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3e0070780 0x7fa3e009cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e5524640 1 -- 172.21.15.5:0/3230056277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d00094d0 con 0x7fa3e00b3140 2023-11-24T06:15:56.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.352+0000 7fa3e5d25640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3e0070780 0x7fa3e009cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:15:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.353+0000 7fa3e5524640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e00b3140 0x7fa3e00acb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa3d002b8a0 tx=0x7fa3d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.353+0000 7fa3d67fc640 1 -- 172.21.15.5:0/3230056277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00329a0 con 0x7fa3e00b3140 2023-11-24T06:15:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.353+0000 7fa3d67fc640 1 -- 172.21.15.5:0/3230056277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d0032b40 con 0x7fa3e00b3140 2023-11-24T06:15:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.353+0000 7fa3d67fc640 1 -- 172.21.15.5:0/3230056277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00453f0 con 0x7fa3e00b3140 2023-11-24T06:15:56.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.353+0000 7fa3e7f89640 1 -- 172.21.15.5:0/3230056277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e00ad8b0 con 0x7fa3e00b3140 2023-11-24T06:15:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.353+0000 7fa3e7f89640 1 -- 172.21.15.5:0/3230056277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e00ade70 con 0x7fa3e00b3140 2023-11-24T06:15:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.354+0000 7fa3e7f89640 1 -- 172.21.15.5:0/3230056277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e00ae060 con 0x7fa3e00b3140 2023-11-24T06:15:56.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.355+0000 7fa3d67fc640 1 -- 172.21.15.5:0/3230056277 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa3d004e450 con 0x7fa3e00b3140 2023-11-24T06:15:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.356+0000 7fa3d67fc640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa3c406ae40 0x7fa3c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.357+0000 7fa3d67fc640 1 -- 172.21.15.5:0/3230056277 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3d00ab9a0 con 0x7fa3e00b3140 2023-11-24T06:15:56.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.357+0000 7fa3e5d25640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa3c406ae40 0x7fa3c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:56.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.357+0000 7fa3e5d25640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa3c406ae40 0x7fa3c406d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00a990 tx=0x7fa3dc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:56.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.361+0000 7fa3d67fc640 1 -- 172.21.15.5:0/3230056277 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3d0076c00 con 0x7fa3e00b3140 2023-11-24T06:15:56.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.545+0000 7fa3e7f89640 1 -- 172.21.15.5:0/3230056277 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa3e0071fb0 con 0x7fa3c406ae40 2023-11-24T06:15:56.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.570+0000 7fa3d67fc640 1 -- 172.21.15.5:0/3230056277 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264440 (secure 0 0 0) 0x7fa3dc00a710 con 0x7fa3c406ae40 2023-11-24T06:15:56.577 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:15:56.578 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-11-24T06:15:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.575+0000 7fa3e7f89640 1 -- 172.21.15.5:0/3230056277 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa3c406ae40 msgr2=0x7fa3c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.575+0000 7fa3e7f89640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa3c406ae40 0x7fa3c406d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00a990 tx=0x7fa3dc008040 comp rx=0 tx=0).stop 2023-11-24T06:15:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.575+0000 7fa3e7f89640 1 -- 172.21.15.5:0/3230056277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e00b3140 msgr2=0x7fa3e00acb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.575+0000 7fa3e7f89640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e00b3140 0x7fa3e00acb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa3d002b8a0 tx=0x7fa3d00046c0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.576+0000 7fa3e7f89640 1 -- 172.21.15.5:0/3230056277 shutdown_connections 2023-11-24T06:15:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.576+0000 7fa3e7f89640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa3c406ae40 0x7fa3c406d300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.576+0000 7fa3e7f89640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3e0160970 0x7fa3e0152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.576+0000 7fa3e7f89640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e00b3140 0x7fa3e00acb30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.576+0000 7fa3e7f89640 1 --2- 172.21.15.5:0/3230056277 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3e0070780 0x7fa3e009cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.576+0000 7fa3e7f89640 1 -- 172.21.15.5:0/3230056277 >> 172.21.15.5:0/3230056277 conn(0x7fa3e00b0d80 msgr2=0x7fa3e0161910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:56.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.576+0000 7fa3e7f89640 1 -- 172.21.15.5:0/3230056277 shutdown_connections 2023-11-24T06:15:56.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.576+0000 7fa3e7f89640 1 -- 172.21.15.5:0/3230056277 wait complete. 2023-11-24T06:15:56.597 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-24T06:15:56.088237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235724,"kb_used_data":2912,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738996,"statfs":{"total":773094113280,"available":772852731904,"internally_reserved":0,"allocated":2981888,"data_stored":1790689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.976976"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185145+0000","last_change":"2023-11-24T06:15:51.826913+0000","last_active":"2023-11-24T06:15:54.185145+0000","last_peered":"2023-11-24T06:15:54.185145+0000","last_clean":"2023-11-24T06:15:54.185145+0000","last_became_active":"2023-11-24T06:15:51.826683+0000","last_became_peered":"2023-11-24T06:15:51.826683+0000","last_unstale":"2023-11-24T06:15:54.185145+0000","last_undegraded":"2023-11-24T06:15:54.185145+0000","last_fullsized":"2023-11-24T06:15:54.185145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:45:59.277130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789469+0000","last_change":"2023-11-24T06:15:51.820105+0000","last_active":"2023-11-24T06:15:52.789469+0000","last_peered":"2023-11-24T06:15:52.789469+0000","last_clean":"2023-11-24T06:15:52.789469+0000","last_became_active":"2023-11-24T06:15:51.819862+0000","last_became_peered":"2023-11-24T06:15:51.819862+0000","last_unstale":"2023-11-24T06:15:52.789469+0000","last_undegraded":"2023-11-24T06:15:52.789469+0000","last_fullsized":"2023-11-24T06:15:52.789469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:58.501546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.824288+0000","last_change":"2023-11-24T06:15:51.817345+0000","last_active":"2023-11-24T06:15:51.824288+0000","last_peered":"2023-11-24T06:15:51.824288+0000","last_clean":"2023-11-24T06:15:51.824288+0000","last_became_active":"2023-11-24T06:15:51.815455+0000","last_became_peered":"2023-11-24T06:15:51.815455+0000","last_unstale":"2023-11-24T06:15:51.824288+0000","last_undegraded":"2023-11-24T06:15:51.824288+0000","last_fullsized":"2023-11-24T06:15:51.824288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:32.279875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793545+0000","last_change":"2023-11-24T06:15:51.811576+0000","last_active":"2023-11-24T06:15:52.793545+0000","last_peered":"2023-11-24T06:15:52.793545+0000","last_clean":"2023-11-24T06:15:52.793545+0000","last_became_active":"2023-11-24T06:15:51.811306+0000","last_became_peered":"2023-11-24T06:15:51.811306+0000","last_unstale":"2023-11-24T06:15:52.793545+0000","last_undegraded":"2023-11-24T06:15:52.793545+0000","last_fullsized":"2023-11-24T06:15:52.793545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:34.521019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184624+0000","last_change":"2023-11-24T06:15:51.821890+0000","last_active":"2023-11-24T06:15:54.184624+0000","last_peered":"2023-11-24T06:15:54.184624+0000","last_clean":"2023-11-24T06:15:54.184624+0000","last_became_active":"2023-11-24T06:15:51.821632+0000","last_became_peered":"2023-11-24T06:15:51.821632+0000","last_unstale":"2023-11-24T06:15:54.184624+0000","last_undegraded":"2023-11-24T06:15:54.184624+0000","last_fullsized":"2023-11-24T06:15:54.184624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:54.384409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792886+0000","last_change":"2023-11-24T06:15:51.828326+0000","last_active":"2023-11-24T06:15:52.792886+0000","last_peered":"2023-11-24T06:15:52.792886+0000","last_clean":"2023-11-24T06:15:52.792886+0000","last_became_active":"2023-11-24T06:15:51.827922+0000","last_became_peered":"2023-11-24T06:15:51.827922+0000","last_unstale":"2023-11-24T06:15:52.792886+0000","last_undegraded":"2023-11-24T06:15:52.792886+0000","last_fullsized":"2023-11-24T06:15:52.792886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:25.825619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183858+0000","last_change":"2023-11-24T06:15:51.825625+0000","last_active":"2023-11-24T06:15:54.183858+0000","last_peered":"2023-11-24T06:15:54.183858+0000","last_clean":"2023-11-24T06:15:54.183858+0000","last_became_active":"2023-11-24T06:15:51.825308+0000","last_became_peered":"2023-11-24T06:15:51.825308+0000","last_unstale":"2023-11-24T06:15:54.183858+0000","last_undegraded":"2023-11-24T06:15:54.183858+0000","last_fullsized":"2023-11-24T06:15:54.183858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:58.501546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844647+0000","last_change":"2023-11-24T06:15:51.817175+0000","last_active":"2023-11-24T06:15:53.844647+0000","last_peered":"2023-11-24T06:15:53.844647+0000","last_clean":"2023-11-24T06:15:53.844647+0000","last_became_active":"2023-11-24T06:15:51.816363+0000","last_became_peered":"2023-11-24T06:15:51.816363+0000","last_unstale":"2023-11-24T06:15:53.844647+0000","last_undegraded":"2023-11-24T06:15:53.844647+0000","last_fullsized":"2023-11-24T06:15:53.844647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:00:51.554978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185142+0000","last_change":"2023-11-24T06:15:51.829604+0000","last_active":"2023-11-24T06:15:54.185142+0000","last_peered":"2023-11-24T06:15:54.185142+0000","last_clean":"2023-11-24T06:15:54.185142+0000","last_became_active":"2023-11-24T06:15:51.829291+0000","last_became_peered":"2023-11-24T06:15:51.829291+0000","last_unstale":"2023-11-24T06:15:54.185142+0000","last_undegraded":"2023-11-24T06:15:54.185142+0000","last_fullsized":"2023-11-24T06:15:54.185142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:57:53.020152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750691+0000","last_change":"2023-11-24T06:15:51.820996+0000","last_active":"2023-11-24T06:15:53.750691+0000","last_peered":"2023-11-24T06:15:53.750691+0000","last_clean":"2023-11-24T06:15:53.750691+0000","last_became_active":"2023-11-24T06:15:51.820018+0000","last_became_peered":"2023-11-24T06:15:51.820018+0000","last_unstale":"2023-11-24T06:15:53.750691+0000","last_undegraded":"2023-11-24T06:15:53.750691+0000","last_fullsized":"2023-11-24T06:15:53.750691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:05.782767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750915+0000","last_change":"2023-11-24T06:15:51.806762+0000","last_active":"2023-11-24T06:15:53.750915+0000","last_peered":"2023-11-24T06:15:53.750915+0000","last_clean":"2023-11-24T06:15:53.750915+0000","last_became_active":"2023-11-24T06:15:51.806476+0000","last_became_peered":"2023-11-24T06:15:51.806476+0000","last_unstale":"2023-11-24T06:15:53.750915+0000","last_undegraded":"2023-11-24T06:15:53.750915+0000","last_fullsized":"2023-11-24T06:15:53.750915+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:25.825619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791243+0000","last_change":"2023-11-24T06:15:51.807959+0000","last_active":"2023-11-24T06:15:52.791243+0000","last_peered":"2023-11-24T06:15:52.791243+0000","last_clean":"2023-11-24T06:15:52.791243+0000","last_became_active":"2023-11-24T06:15:51.807625+0000","last_became_peered":"2023-11-24T06:15:51.807625+0000","last_unstale":"2023-11-24T06:15:52.791243+0000","last_undegraded":"2023-11-24T06:15:52.791243+0000","last_fullsized":"2023-11-24T06:15:52.791243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:37.913398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845276+0000","last_change":"2023-11-24T06:15:51.820393+0000","last_active":"2023-11-24T06:15:53.845276+0000","last_peered":"2023-11-24T06:15:53.845276+0000","last_clean":"2023-11-24T06:15:53.845276+0000","last_became_active":"2023-11-24T06:15:51.820092+0000","last_became_peered":"2023-11-24T06:15:51.820092+0000","last_unstale":"2023-11-24T06:15:53.845276+0000","last_undegraded":"2023-11-24T06:15:53.845276+0000","last_fullsized":"2023-11-24T06:15:53.845276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:16.273773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.824842+0000","last_change":"2023-11-24T06:15:51.822418+0000","last_active":"2023-11-24T06:15:51.824842+0000","last_peered":"2023-11-24T06:15:51.824842+0000","last_clean":"2023-11-24T06:15:51.824842+0000","last_became_active":"2023-11-24T06:15:51.821162+0000","last_became_peered":"2023-11-24T06:15:51.821162+0000","last_unstale":"2023-11-24T06:15:51.824842+0000","last_undegraded":"2023-11-24T06:15:51.824842+0000","last_fullsized":"2023-11-24T06:15:51.824842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:43.354531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788580+0000","last_change":"2023-11-24T06:15:51.808930+0000","last_active":"2023-11-24T06:15:52.788580+0000","last_peered":"2023-11-24T06:15:52.788580+0000","last_clean":"2023-11-24T06:15:52.788580+0000","last_became_active":"2023-11-24T06:15:51.808593+0000","last_became_peered":"2023-11-24T06:15:51.808593+0000","last_unstale":"2023-11-24T06:15:52.788580+0000","last_undegraded":"2023-11-24T06:15:52.788580+0000","last_fullsized":"2023-11-24T06:15:52.788580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:43.291318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792334+0000","last_change":"2023-11-24T06:15:51.829868+0000","last_active":"2023-11-24T06:15:52.792334+0000","last_peered":"2023-11-24T06:15:52.792334+0000","last_clean":"2023-11-24T06:15:52.792334+0000","last_became_active":"2023-11-24T06:15:51.828875+0000","last_became_peered":"2023-11-24T06:15:51.828875+0000","last_unstale":"2023-11-24T06:15:52.792334+0000","last_undegraded":"2023-11-24T06:15:52.792334+0000","last_fullsized":"2023-11-24T06:15:52.792334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:32.225953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.794592+0000","last_change":"2023-11-24T06:15:51.814343+0000","last_active":"2023-11-24T06:15:52.794592+0000","last_peered":"2023-11-24T06:15:52.794592+0000","last_clean":"2023-11-24T06:15:52.794592+0000","last_became_active":"2023-11-24T06:15:51.814003+0000","last_became_peered":"2023-11-24T06:15:51.814003+0000","last_unstale":"2023-11-24T06:15:52.794592+0000","last_undegraded":"2023-11-24T06:15:52.794592+0000","last_fullsized":"2023-11-24T06:15:52.794592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:26:09.975671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.139925+0000","last_change":"2023-11-24T06:15:51.815884+0000","last_active":"2023-11-24T06:15:53.139925+0000","last_peered":"2023-11-24T06:15:53.139925+0000","last_clean":"2023-11-24T06:15:53.139925+0000","last_became_active":"2023-11-24T06:15:51.814549+0000","last_became_peered":"2023-11-24T06:15:51.814549+0000","last_unstale":"2023-11-24T06:15:53.139925+0000","last_undegraded":"2023-11-24T06:15:53.139925+0000","last_fullsized":"2023-11-24T06:15:53.139925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:43.354531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185458+0000","last_change":"2023-11-24T06:15:51.823265+0000","last_active":"2023-11-24T06:15:54.185458+0000","last_peered":"2023-11-24T06:15:54.185458+0000","last_clean":"2023-11-24T06:15:54.185458+0000","last_became_active":"2023-11-24T06:15:51.823026+0000","last_became_peered":"2023-11-24T06:15:51.823026+0000","last_unstale":"2023-11-24T06:15:54.185458+0000","last_undegraded":"2023-11-24T06:15:54.185458+0000","last_fullsized":"2023-11-24T06:15:54.185458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:48.210729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750151+0000","last_change":"2023-11-24T06:15:51.818614+0000","last_active":"2023-11-24T06:15:53.750151+0000","last_peered":"2023-11-24T06:15:53.750151+0000","last_clean":"2023-11-24T06:15:53.750151+0000","last_became_active":"2023-11-24T06:15:51.818250+0000","last_became_peered":"2023-11-24T06:15:51.818250+0000","last_unstale":"2023-11-24T06:15:53.750151+0000","last_undegraded":"2023-11-24T06:15:53.750151+0000","last_fullsized":"2023-11-24T06:15:53.750151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:07.460859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.824083+0000","last_change":"2023-11-24T06:15:51.815798+0000","last_active":"2023-11-24T06:15:51.824083+0000","last_peered":"2023-11-24T06:15:51.824083+0000","last_clean":"2023-11-24T06:15:51.824083+0000","last_became_active":"2023-11-24T06:15:51.815531+0000","last_became_peered":"2023-11-24T06:15:51.815531+0000","last_unstale":"2023-11-24T06:15:51.824083+0000","last_undegraded":"2023-11-24T06:15:51.824083+0000","last_fullsized":"2023-11-24T06:15:51.824083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:08.205306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.794291+0000","last_change":"2023-11-24T06:15:51.813391+0000","last_active":"2023-11-24T06:15:52.794291+0000","last_peered":"2023-11-24T06:15:52.794291+0000","last_clean":"2023-11-24T06:15:52.794291+0000","last_became_active":"2023-11-24T06:15:51.813066+0000","last_became_peered":"2023-11-24T06:15:51.813066+0000","last_unstale":"2023-11-24T06:15:52.794291+0000","last_undegraded":"2023-11-24T06:15:52.794291+0000","last_fullsized":"2023-11-24T06:15:52.794291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:54:56.970614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844721+0000","last_change":"2023-11-24T06:15:51.812820+0000","last_active":"2023-11-24T06:15:53.844721+0000","last_peered":"2023-11-24T06:15:53.844721+0000","last_clean":"2023-11-24T06:15:53.844721+0000","last_became_active":"2023-11-24T06:15:51.812517+0000","last_became_peered":"2023-11-24T06:15:51.812517+0000","last_unstale":"2023-11-24T06:15:53.844721+0000","last_undegraded":"2023-11-24T06:15:53.844721+0000","last_fullsized":"2023-11-24T06:15:53.844721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:25.102989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792262+0000","last_change":"2023-11-24T06:15:51.808797+0000","last_active":"2023-11-24T06:15:52.792262+0000","last_peered":"2023-11-24T06:15:52.792262+0000","last_clean":"2023-11-24T06:15:52.792262+0000","last_became_active":"2023-11-24T06:15:51.808481+0000","last_became_peered":"2023-11-24T06:15:51.808481+0000","last_unstale":"2023-11-24T06:15:52.792262+0000","last_undegraded":"2023-11-24T06:15:52.792262+0000","last_fullsized":"2023-11-24T06:15:52.792262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:05.782767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845642+0000","last_change":"2023-11-24T06:15:51.818331+0000","last_active":"2023-11-24T06:15:53.845642+0000","last_peered":"2023-11-24T06:15:53.845642+0000","last_clean":"2023-11-24T06:15:53.845642+0000","last_became_active":"2023-11-24T06:15:51.818011+0000","last_became_peered":"2023-11-24T06:15:51.818011+0000","last_unstale":"2023-11-24T06:15:53.845642+0000","last_undegraded":"2023-11-24T06:15:53.845642+0000","last_fullsized":"2023-11-24T06:15:53.845642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:58.501546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750817+0000","last_change":"2023-11-24T06:15:51.819933+0000","last_active":"2023-11-24T06:15:53.750817+0000","last_peered":"2023-11-24T06:15:53.750817+0000","last_clean":"2023-11-24T06:15:53.750817+0000","last_became_active":"2023-11-24T06:15:51.819592+0000","last_became_peered":"2023-11-24T06:15:51.819592+0000","last_unstale":"2023-11-24T06:15:53.750817+0000","last_undegraded":"2023-11-24T06:15:53.750817+0000","last_fullsized":"2023-11-24T06:15:53.750817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:32.225953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788974+0000","last_change":"2023-11-24T06:15:51.810396+0000","last_active":"2023-11-24T06:15:52.788974+0000","last_peered":"2023-11-24T06:15:52.788974+0000","last_clean":"2023-11-24T06:15:52.788974+0000","last_became_active":"2023-11-24T06:15:51.809939+0000","last_became_peered":"2023-11-24T06:15:51.809939+0000","last_unstale":"2023-11-24T06:15:52.788974+0000","last_undegraded":"2023-11-24T06:15:52.788974+0000","last_fullsized":"2023-11-24T06:15:52.788974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:16.273773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845858+0000","last_change":"2023-11-24T06:15:51.818877+0000","last_active":"2023-11-24T06:15:53.845858+0000","last_peered":"2023-11-24T06:15:53.845858+0000","last_clean":"2023-11-24T06:15:53.845858+0000","last_became_active":"2023-11-24T06:15:51.818068+0000","last_became_peered":"2023-11-24T06:15:51.818068+0000","last_unstale":"2023-11-24T06:15:53.845858+0000","last_undegraded":"2023-11-24T06:15:53.845858+0000","last_fullsized":"2023-11-24T06:15:53.845858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:54.384409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.825444+0000","last_change":"2023-11-24T06:15:51.821479+0000","last_active":"2023-11-24T06:15:51.825444+0000","last_peered":"2023-11-24T06:15:51.825444+0000","last_clean":"2023-11-24T06:15:51.825444+0000","last_became_active":"2023-11-24T06:15:51.820978+0000","last_became_peered":"2023-11-24T06:15:51.820978+0000","last_unstale":"2023-11-24T06:15:51.825444+0000","last_undegraded":"2023-11-24T06:15:51.825444+0000","last_fullsized":"2023-11-24T06:15:51.825444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:08.846287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793272+0000","last_change":"2023-11-24T06:15:51.808157+0000","last_active":"2023-11-24T06:15:52.793272+0000","last_peered":"2023-11-24T06:15:52.793272+0000","last_clean":"2023-11-24T06:15:52.793272+0000","last_became_active":"2023-11-24T06:15:51.807868+0000","last_became_peered":"2023-11-24T06:15:51.807868+0000","last_unstale":"2023-11-24T06:15:52.793272+0000","last_undegraded":"2023-11-24T06:15:52.793272+0000","last_fullsized":"2023-11-24T06:15:52.793272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:41:05.120915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.140134+0000","last_change":"2023-11-24T06:15:51.809534+0000","last_active":"2023-11-24T06:15:53.140134+0000","last_peered":"2023-11-24T06:15:53.140134+0000","last_clean":"2023-11-24T06:15:53.140134+0000","last_became_active":"2023-11-24T06:15:51.809037+0000","last_became_peered":"2023-11-24T06:15:51.809037+0000","last_unstale":"2023-11-24T06:15:53.140134+0000","last_undegraded":"2023-11-24T06:15:53.140134+0000","last_fullsized":"2023-11-24T06:15:53.140134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:38.032032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750282+0000","last_change":"2023-11-24T06:15:51.825619+0000","last_active":"2023-11-24T06:15:53.750282+0000","last_peered":"2023-11-24T06:15:53.750282+0000","last_clean":"2023-11-24T06:15:53.750282+0000","last_became_active":"2023-11-24T06:15:51.825149+0000","last_became_peered":"2023-11-24T06:15:51.825149+0000","last_unstale":"2023-11-24T06:15:53.750282+0000","last_undegraded":"2023-11-24T06:15:53.750282+0000","last_fullsized":"2023-11-24T06:15:53.750282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:12:12.520019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791884+0000","last_change":"2023-11-24T06:15:51.810514+0000","last_active":"2023-11-24T06:15:52.791884+0000","last_peered":"2023-11-24T06:15:52.791884+0000","last_clean":"2023-11-24T06:15:52.791884+0000","last_became_active":"2023-11-24T06:15:51.807943+0000","last_became_peered":"2023-11-24T06:15:51.807943+0000","last_unstale":"2023-11-24T06:15:52.791884+0000","last_undegraded":"2023-11-24T06:15:52.791884+0000","last_fullsized":"2023-11-24T06:15:52.791884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:19.993397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845626+0000","last_change":"2023-11-24T06:15:51.164865+0000","last_active":"2023-11-24T06:15:53.845626+0000","last_peered":"2023-11-24T06:15:53.845626+0000","last_clean":"2023-11-24T06:15:53.845626+0000","last_became_active":"2023-11-24T06:15:51.164524+0000","last_became_peered":"2023-11-24T06:15:51.164524+0000","last_unstale":"2023-11-24T06:15:53.845626+0000","last_undegraded":"2023-11-24T06:15:53.845626+0000","last_fullsized":"2023-11-24T06:15:53.845626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:02.689172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.825431+0000","last_change":"2023-11-24T06:15:51.818737+0000","last_active":"2023-11-24T06:15:51.825431+0000","last_peered":"2023-11-24T06:15:51.825431+0000","last_clean":"2023-11-24T06:15:51.825431+0000","last_became_active":"2023-11-24T06:15:51.817354+0000","last_became_peered":"2023-11-24T06:15:51.817354+0000","last_unstale":"2023-11-24T06:15:51.825431+0000","last_undegraded":"2023-11-24T06:15:51.825431+0000","last_fullsized":"2023-11-24T06:15:51.825431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:27:22.945202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749981+0000","last_change":"2023-11-24T06:15:51.169166+0000","last_active":"2023-11-24T06:15:53.749981+0000","last_peered":"2023-11-24T06:15:53.749981+0000","last_clean":"2023-11-24T06:15:53.749981+0000","last_became_active":"2023-11-24T06:15:51.168863+0000","last_became_peered":"2023-11-24T06:15:51.168863+0000","last_unstale":"2023-11-24T06:15:53.749981+0000","last_undegraded":"2023-11-24T06:15:53.749981+0000","last_fullsized":"2023-11-24T06:15:53.749981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:16.146869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844773+0000","last_change":"2023-11-24T06:15:51.165957+0000","last_active":"2023-11-24T06:15:53.844773+0000","last_peered":"2023-11-24T06:15:53.844773+0000","last_clean":"2023-11-24T06:15:53.844773+0000","last_became_active":"2023-11-24T06:15:51.165575+0000","last_became_peered":"2023-11-24T06:15:51.165575+0000","last_unstale":"2023-11-24T06:15:53.844773+0000","last_undegraded":"2023-11-24T06:15:53.844773+0000","last_fullsized":"2023-11-24T06:15:53.844773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:09.461899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844747+0000","last_change":"2023-11-24T06:15:51.818508+0000","last_active":"2023-11-24T06:15:53.844747+0000","last_peered":"2023-11-24T06:15:53.844747+0000","last_clean":"2023-11-24T06:15:53.844747+0000","last_became_active":"2023-11-24T06:15:51.818158+0000","last_became_peered":"2023-11-24T06:15:51.818158+0000","last_unstale":"2023-11-24T06:15:53.844747+0000","last_undegraded":"2023-11-24T06:15:53.844747+0000","last_fullsized":"2023-11-24T06:15:53.844747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:43.291318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792534+0000","last_change":"2023-11-24T06:15:51.168437+0000","last_active":"2023-11-24T06:15:52.792534+0000","last_peered":"2023-11-24T06:15:52.792534+0000","last_clean":"2023-11-24T06:15:52.792534+0000","last_became_active":"2023-11-24T06:15:51.162079+0000","last_became_peered":"2023-11-24T06:15:51.162079+0000","last_unstale":"2023-11-24T06:15:52.792534+0000","last_undegraded":"2023-11-24T06:15:52.792534+0000","last_fullsized":"2023-11-24T06:15:52.792534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:07.834230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788318+0000","last_change":"2023-11-24T06:15:51.811023+0000","last_active":"2023-11-24T06:15:52.788318+0000","last_peered":"2023-11-24T06:15:52.788318+0000","last_clean":"2023-11-24T06:15:52.788318+0000","last_became_active":"2023-11-24T06:15:51.810368+0000","last_became_peered":"2023-11-24T06:15:51.810368+0000","last_unstale":"2023-11-24T06:15:52.788318+0000","last_undegraded":"2023-11-24T06:15:52.788318+0000","last_fullsized":"2023-11-24T06:15:52.788318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:25.102989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750531+0000","last_change":"2023-11-24T06:15:51.175246+0000","last_active":"2023-11-24T06:15:53.750531+0000","last_peered":"2023-11-24T06:15:53.750531+0000","last_clean":"2023-11-24T06:15:53.750531+0000","last_became_active":"2023-11-24T06:15:51.174941+0000","last_became_peered":"2023-11-24T06:15:51.174941+0000","last_unstale":"2023-11-24T06:15:53.750531+0000","last_undegraded":"2023-11-24T06:15:53.750531+0000","last_fullsized":"2023-11-24T06:15:53.750531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:48.317425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793290+0000","last_change":"2023-11-24T06:15:51.822074+0000","last_active":"2023-11-24T06:15:52.793290+0000","last_peered":"2023-11-24T06:15:52.793290+0000","last_clean":"2023-11-24T06:15:52.793290+0000","last_became_active":"2023-11-24T06:15:51.821749+0000","last_became_peered":"2023-11-24T06:15:51.821749+0000","last_unstale":"2023-11-24T06:15:52.793290+0000","last_undegraded":"2023-11-24T06:15:52.793290+0000","last_fullsized":"2023-11-24T06:15:52.793290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:47:30.989438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793436+0000","last_change":"2023-11-24T06:15:51.175128+0000","last_active":"2023-11-24T06:15:52.793436+0000","last_peered":"2023-11-24T06:15:52.793436+0000","last_clean":"2023-11-24T06:15:52.793436+0000","last_became_active":"2023-11-24T06:15:51.173422+0000","last_became_peered":"2023-11-24T06:15:51.173422+0000","last_unstale":"2023-11-24T06:15:52.793436+0000","last_undegraded":"2023-11-24T06:15:52.793436+0000","last_fullsized":"2023-11-24T06:15:52.793436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:12:11.507962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.822454+0000","last_change":"2023-11-24T06:15:51.814210+0000","last_active":"2023-11-24T06:15:51.822454+0000","last_peered":"2023-11-24T06:15:51.822454+0000","last_clean":"2023-11-24T06:15:51.822454+0000","last_became_active":"2023-11-24T06:15:51.813951+0000","last_became_peered":"2023-11-24T06:15:51.813951+0000","last_unstale":"2023-11-24T06:15:51.822454+0000","last_undegraded":"2023-11-24T06:15:51.822454+0000","last_fullsized":"2023-11-24T06:15:51.822454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:48.919036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789082+0000","last_change":"2023-11-24T06:15:51.176266+0000","last_active":"2023-11-24T06:15:52.789082+0000","last_peered":"2023-11-24T06:15:52.789082+0000","last_clean":"2023-11-24T06:15:52.789082+0000","last_became_active":"2023-11-24T06:15:51.175918+0000","last_became_peered":"2023-11-24T06:15:51.175918+0000","last_unstale":"2023-11-24T06:15:52.789082+0000","last_undegraded":"2023-11-24T06:15:52.789082+0000","last_fullsized":"2023-11-24T06:15:52.789082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:18.981340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791650+0000","last_change":"2023-11-24T06:15:51.819556+0000","last_active":"2023-11-24T06:15:52.791650+0000","last_peered":"2023-11-24T06:15:52.791650+0000","last_clean":"2023-11-24T06:15:52.791650+0000","last_became_active":"2023-11-24T06:15:51.818398+0000","last_became_peered":"2023-11-24T06:15:51.818398+0000","last_unstale":"2023-11-24T06:15:52.791650+0000","last_undegraded":"2023-11-24T06:15:52.791650+0000","last_fullsized":"2023-11-24T06:15:52.791650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:41.065020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750080+0000","last_change":"2023-11-24T06:15:51.169595+0000","last_active":"2023-11-24T06:15:53.750080+0000","last_peered":"2023-11-24T06:15:53.750080+0000","last_clean":"2023-11-24T06:15:53.750080+0000","last_became_active":"2023-11-24T06:15:51.167755+0000","last_became_peered":"2023-11-24T06:15:51.167755+0000","last_unstale":"2023-11-24T06:15:53.750080+0000","last_undegraded":"2023-11-24T06:15:53.750080+0000","last_fullsized":"2023-11-24T06:15:53.750080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:37.019975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184774+0000","last_change":"2023-11-24T06:15:51.821305+0000","last_active":"2023-11-24T06:15:54.184774+0000","last_peered":"2023-11-24T06:15:54.184774+0000","last_clean":"2023-11-24T06:15:54.184774+0000","last_became_active":"2023-11-24T06:15:51.821061+0000","last_became_peered":"2023-11-24T06:15:51.821061+0000","last_unstale":"2023-11-24T06:15:54.184774+0000","last_undegraded":"2023-11-24T06:15:54.184774+0000","last_fullsized":"2023-11-24T06:15:54.184774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:24.298719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.158012+0000","last_change":"2023-11-24T06:15:51.150730+0000","last_active":"2023-11-24T06:15:52.158012+0000","last_peered":"2023-11-24T06:15:52.158012+0000","last_clean":"2023-11-24T06:15:52.158012+0000","last_became_active":"2023-11-24T06:15:51.147375+0000","last_became_peered":"2023-11-24T06:15:51.147375+0000","last_unstale":"2023-11-24T06:15:52.158012+0000","last_undegraded":"2023-11-24T06:15:52.158012+0000","last_fullsized":"2023-11-24T06:15:52.158012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:24:38.346475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.824860+0000","last_change":"2023-11-24T06:15:51.822499+0000","last_active":"2023-11-24T06:15:51.824860+0000","last_peered":"2023-11-24T06:15:51.824860+0000","last_clean":"2023-11-24T06:15:51.824860+0000","last_became_active":"2023-11-24T06:15:51.822192+0000","last_became_peered":"2023-11-24T06:15:51.822192+0000","last_unstale":"2023-11-24T06:15:51.824860+0000","last_undegraded":"2023-11-24T06:15:51.824860+0000","last_fullsized":"2023-11-24T06:15:51.824860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:47:59.130691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793561+0000","last_change":"2023-11-24T06:15:51.174234+0000","last_active":"2023-11-24T06:15:52.793561+0000","last_peered":"2023-11-24T06:15:52.793561+0000","last_clean":"2023-11-24T06:15:52.793561+0000","last_became_active":"2023-11-24T06:15:51.173741+0000","last_became_peered":"2023-11-24T06:15:51.173741+0000","last_unstale":"2023-11-24T06:15:52.793561+0000","last_undegraded":"2023-11-24T06:15:52.793561+0000","last_fullsized":"2023-11-24T06:15:52.793561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:16.146869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184654+0000","last_change":"2023-11-24T06:15:51.828660+0000","last_active":"2023-11-24T06:15:54.184654+0000","last_peered":"2023-11-24T06:15:54.184654+0000","last_clean":"2023-11-24T06:15:54.184654+0000","last_became_active":"2023-11-24T06:15:51.828176+0000","last_became_peered":"2023-11-24T06:15:51.828176+0000","last_unstale":"2023-11-24T06:15:54.184654+0000","last_undegraded":"2023-11-24T06:15:54.184654+0000","last_fullsized":"2023-11-24T06:15:54.184654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:47.291087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749753+0000","last_change":"2023-11-24T06:15:51.171268+0000","last_active":"2023-11-24T06:15:53.749753+0000","last_peered":"2023-11-24T06:15:53.749753+0000","last_clean":"2023-11-24T06:15:53.749753+0000","last_became_active":"2023-11-24T06:15:51.170887+0000","last_became_peered":"2023-11-24T06:15:51.170887+0000","last_unstale":"2023-11-24T06:15:53.749753+0000","last_undegraded":"2023-11-24T06:15:53.749753+0000","last_fullsized":"2023-11-24T06:15:53.749753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:24:38.346475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749967+0000","last_change":"2023-11-24T06:15:51.825705+0000","last_active":"2023-11-24T06:15:53.749967+0000","last_peered":"2023-11-24T06:15:53.749967+0000","last_clean":"2023-11-24T06:15:53.749967+0000","last_became_active":"2023-11-24T06:15:51.825372+0000","last_became_peered":"2023-11-24T06:15:51.825372+0000","last_unstale":"2023-11-24T06:15:53.749967+0000","last_undegraded":"2023-11-24T06:15:53.749967+0000","last_fullsized":"2023-11-24T06:15:53.749967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:57:57.197250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.139651+0000","last_change":"2023-11-24T06:15:51.155610+0000","last_active":"2023-11-24T06:15:53.139651+0000","last_peered":"2023-11-24T06:15:53.139651+0000","last_clean":"2023-11-24T06:15:53.139651+0000","last_became_active":"2023-11-24T06:15:51.155318+0000","last_became_peered":"2023-11-24T06:15:51.155318+0000","last_unstale":"2023-11-24T06:15:53.139651+0000","last_undegraded":"2023-11-24T06:15:53.139651+0000","last_fullsized":"2023-11-24T06:15:53.139651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:07.193249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183869+0000","last_change":"2023-11-24T06:15:51.825462+0000","last_active":"2023-11-24T06:15:54.183869+0000","last_peered":"2023-11-24T06:15:54.183869+0000","last_clean":"2023-11-24T06:15:54.183869+0000","last_became_active":"2023-11-24T06:15:51.825148+0000","last_became_peered":"2023-11-24T06:15:51.825148+0000","last_unstale":"2023-11-24T06:15:54.183869+0000","last_undegraded":"2023-11-24T06:15:54.183869+0000","last_fullsized":"2023-11-24T06:15:54.183869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:46.484813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.158814+0000","last_change":"2023-11-24T06:15:51.151498+0000","last_active":"2023-11-24T06:15:52.158814+0000","last_peered":"2023-11-24T06:15:52.158814+0000","last_clean":"2023-11-24T06:15:52.158814+0000","last_became_active":"2023-11-24T06:15:51.151116+0000","last_became_peered":"2023-11-24T06:15:51.151116+0000","last_unstale":"2023-11-24T06:15:52.158814+0000","last_undegraded":"2023-11-24T06:15:52.158814+0000","last_fullsized":"2023-11-24T06:15:52.158814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:32.153844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184490+0000","last_change":"2023-11-24T06:15:51.826667+0000","last_active":"2023-11-24T06:15:54.184490+0000","last_peered":"2023-11-24T06:15:54.184490+0000","last_clean":"2023-11-24T06:15:54.184490+0000","last_became_active":"2023-11-24T06:15:51.826286+0000","last_became_peered":"2023-11-24T06:15:51.826286+0000","last_unstale":"2023-11-24T06:15:54.184490+0000","last_undegraded":"2023-11-24T06:15:54.184490+0000","last_fullsized":"2023-11-24T06:15:54.184490+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:27.816377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.140108+0000","last_change":"2023-11-24T06:15:51.156776+0000","last_active":"2023-11-24T06:15:53.140108+0000","last_peered":"2023-11-24T06:15:53.140108+0000","last_clean":"2023-11-24T06:15:53.140108+0000","last_became_active":"2023-11-24T06:15:51.156313+0000","last_became_peered":"2023-11-24T06:15:51.156313+0000","last_unstale":"2023-11-24T06:15:53.140108+0000","last_undegraded":"2023-11-24T06:15:53.140108+0000","last_fullsized":"2023-11-24T06:15:53.140108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:47:58.118634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185599+0000","last_change":"2023-11-24T06:15:51.825443+0000","last_active":"2023-11-24T06:15:54.185599+0000","last_peered":"2023-11-24T06:15:54.185599+0000","last_clean":"2023-11-24T06:15:54.185599+0000","last_became_active":"2023-11-24T06:15:51.825149+0000","last_became_peered":"2023-11-24T06:15:51.825149+0000","last_unstale":"2023-11-24T06:15:54.185599+0000","last_undegraded":"2023-11-24T06:15:54.185599+0000","last_fullsized":"2023-11-24T06:15:54.185599+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:37.832748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789200+0000","last_change":"2023-11-24T06:15:51.158822+0000","last_active":"2023-11-24T06:15:52.789200+0000","last_peered":"2023-11-24T06:15:52.789200+0000","last_clean":"2023-11-24T06:15:52.789200+0000","last_became_active":"2023-11-24T06:15:51.158544+0000","last_became_peered":"2023-11-24T06:15:51.158544+0000","last_unstale":"2023-11-24T06:15:52.789200+0000","last_undegraded":"2023-11-24T06:15:52.789200+0000","last_fullsized":"2023-11-24T06:15:52.789200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:02.689172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.794289+0000","last_change":"2023-11-24T06:15:51.808175+0000","last_active":"2023-11-24T06:15:52.794289+0000","last_peered":"2023-11-24T06:15:52.794289+0000","last_clean":"2023-11-24T06:15:52.794289+0000","last_became_active":"2023-11-24T06:15:51.807904+0000","last_became_peered":"2023-11-24T06:15:51.807904+0000","last_unstale":"2023-11-24T06:15:52.794289+0000","last_undegraded":"2023-11-24T06:15:52.794289+0000","last_fullsized":"2023-11-24T06:15:52.794289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:54.525459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.139914+0000","last_change":"2023-11-24T06:15:51.175398+0000","last_active":"2023-11-24T06:15:53.139914+0000","last_peered":"2023-11-24T06:15:53.139914+0000","last_clean":"2023-11-24T06:15:53.139914+0000","last_became_active":"2023-11-24T06:15:51.175039+0000","last_became_peered":"2023-11-24T06:15:51.175039+0000","last_unstale":"2023-11-24T06:15:53.139914+0000","last_undegraded":"2023-11-24T06:15:53.139914+0000","last_fullsized":"2023-11-24T06:15:53.139914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:07.834230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185074+0000","last_change":"2023-11-24T06:15:51.829803+0000","last_active":"2023-11-24T06:15:54.185074+0000","last_peered":"2023-11-24T06:15:54.185074+0000","last_clean":"2023-11-24T06:15:54.185074+0000","last_became_active":"2023-11-24T06:15:51.829417+0000","last_became_peered":"2023-11-24T06:15:51.829417+0000","last_unstale":"2023-11-24T06:15:54.185074+0000","last_undegraded":"2023-11-24T06:15:54.185074+0000","last_fullsized":"2023-11-24T06:15:54.185074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:55:04.397371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.810683+0000","last_change":"2023-11-24T06:15:42.021608+0000","last_active":"2023-11-24T06:15:51.810683+0000","last_peered":"2023-11-24T06:15:51.810683+0000","last_clean":"2023-11-24T06:15:51.810683+0000","last_became_active":"2023-11-24T06:15:42.018847+0000","last_became_peered":"2023-11-24T06:15:42.018847+0000","last_unstale":"2023-11-24T06:15:51.810683+0000","last_undegraded":"2023-11-24T06:15:51.810683+0000","last_fullsized":"2023-11-24T06:15:51.810683+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_clean_scrub_stamp":"2023-11-24T06:15:40.694368+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:41.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792437+0000","last_change":"2023-11-24T06:15:51.161936+0000","last_active":"2023-11-24T06:15:52.792437+0000","last_peered":"2023-11-24T06:15:52.792437+0000","last_clean":"2023-11-24T06:15:52.792437+0000","last_became_active":"2023-11-24T06:15:51.161566+0000","last_became_peered":"2023-11-24T06:15:51.161566+0000","last_unstale":"2023-11-24T06:15:52.792437+0000","last_undegraded":"2023-11-24T06:15:52.792437+0000","last_fullsized":"2023-11-24T06:15:52.792437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:42.342474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845108+0000","last_change":"2023-11-24T06:15:51.816547+0000","last_active":"2023-11-24T06:15:53.845108+0000","last_peered":"2023-11-24T06:15:53.845108+0000","last_clean":"2023-11-24T06:15:53.845108+0000","last_became_active":"2023-11-24T06:15:51.816231+0000","last_became_peered":"2023-11-24T06:15:51.816231+0000","last_unstale":"2023-11-24T06:15:53.845108+0000","last_undegraded":"2023-11-24T06:15:53.845108+0000","last_fullsized":"2023-11-24T06:15:53.845108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:21:49.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183719+0000","last_change":"2023-11-24T06:15:51.164362+0000","last_active":"2023-11-24T06:15:54.183719+0000","last_peered":"2023-11-24T06:15:54.183719+0000","last_clean":"2023-11-24T06:15:54.183719+0000","last_became_active":"2023-11-24T06:15:51.163606+0000","last_became_peered":"2023-11-24T06:15:51.163606+0000","last_unstale":"2023-11-24T06:15:54.183719+0000","last_undegraded":"2023-11-24T06:15:54.183719+0000","last_fullsized":"2023-11-24T06:15:54.183719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:36.901341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.824729+0000","last_change":"2023-11-24T06:15:51.821396+0000","last_active":"2023-11-24T06:15:51.824729+0000","last_peered":"2023-11-24T06:15:51.824729+0000","last_clean":"2023-11-24T06:15:51.824729+0000","last_became_active":"2023-11-24T06:15:51.821096+0000","last_became_peered":"2023-11-24T06:15:51.821096+0000","last_unstale":"2023-11-24T06:15:51.824729+0000","last_undegraded":"2023-11-24T06:15:51.824729+0000","last_fullsized":"2023-11-24T06:15:51.824729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:38.032032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.140207+0000","last_change":"2023-11-24T06:15:51.156378+0000","last_active":"2023-11-24T06:15:53.140207+0000","last_peered":"2023-11-24T06:15:53.140207+0000","last_clean":"2023-11-24T06:15:53.140207+0000","last_became_active":"2023-11-24T06:15:51.156003+0000","last_became_peered":"2023-11-24T06:15:51.156003+0000","last_unstale":"2023-11-24T06:15:53.140207+0000","last_undegraded":"2023-11-24T06:15:53.140207+0000","last_fullsized":"2023-11-24T06:15:53.140207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:31.267818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788625+0000","last_change":"2023-11-24T06:15:51.810053+0000","last_active":"2023-11-24T06:15:52.788625+0000","last_peered":"2023-11-24T06:15:52.788625+0000","last_clean":"2023-11-24T06:15:52.788625+0000","last_became_active":"2023-11-24T06:15:51.806554+0000","last_became_peered":"2023-11-24T06:15:51.806554+0000","last_unstale":"2023-11-24T06:15:52.788625+0000","last_undegraded":"2023-11-24T06:15:52.788625+0000","last_fullsized":"2023-11-24T06:15:52.788625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:01.976941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793809+0000","last_change":"2023-11-24T06:15:51.170982+0000","last_active":"2023-11-24T06:15:52.793809+0000","last_peered":"2023-11-24T06:15:52.793809+0000","last_clean":"2023-11-24T06:15:52.793809+0000","last_became_active":"2023-11-24T06:15:51.170654+0000","last_became_peered":"2023-11-24T06:15:51.170654+0000","last_unstale":"2023-11-24T06:15:52.793809+0000","last_undegraded":"2023-11-24T06:15:52.793809+0000","last_fullsized":"2023-11-24T06:15:52.793809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:48.317425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792676+0000","last_change":"2023-11-24T06:15:51.823915+0000","last_active":"2023-11-24T06:15:52.792676+0000","last_peered":"2023-11-24T06:15:52.792676+0000","last_clean":"2023-11-24T06:15:52.792676+0000","last_became_active":"2023-11-24T06:15:51.823654+0000","last_became_peered":"2023-11-24T06:15:51.823654+0000","last_unstale":"2023-11-24T06:15:52.792676+0000","last_undegraded":"2023-11-24T06:15:52.792676+0000","last_fullsized":"2023-11-24T06:15:52.792676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:17:01.717621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789034+0000","last_change":"2023-11-24T06:15:51.154171+0000","last_active":"2023-11-24T06:15:52.789034+0000","last_peered":"2023-11-24T06:15:52.789034+0000","last_clean":"2023-11-24T06:15:52.789034+0000","last_became_active":"2023-11-24T06:15:51.153885+0000","last_became_peered":"2023-11-24T06:15:51.153885+0000","last_unstale":"2023-11-24T06:15:52.789034+0000","last_undegraded":"2023-11-24T06:15:52.789034+0000","last_fullsized":"2023-11-24T06:15:52.789034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:36:33.590350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793463+0000","last_change":"2023-11-24T06:15:51.809270+0000","last_active":"2023-11-24T06:15:52.793463+0000","last_peered":"2023-11-24T06:15:52.793463+0000","last_clean":"2023-11-24T06:15:52.793463+0000","last_became_active":"2023-11-24T06:15:51.808994+0000","last_became_peered":"2023-11-24T06:15:51.808994+0000","last_unstale":"2023-11-24T06:15:52.793463+0000","last_undegraded":"2023-11-24T06:15:52.793463+0000","last_fullsized":"2023-11-24T06:15:52.793463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:54.543519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185536+0000","last_change":"2023-11-24T06:15:51.166677+0000","last_active":"2023-11-24T06:15:54.185536+0000","last_peered":"2023-11-24T06:15:54.185536+0000","last_clean":"2023-11-24T06:15:54.185536+0000","last_became_active":"2023-11-24T06:15:51.164374+0000","last_became_peered":"2023-11-24T06:15:51.164374+0000","last_unstale":"2023-11-24T06:15:54.185536+0000","last_undegraded":"2023-11-24T06:15:54.185536+0000","last_fullsized":"2023-11-24T06:15:54.185536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:00:50.542922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788766+0000","last_change":"2023-11-24T06:15:51.808320+0000","last_active":"2023-11-24T06:15:52.788766+0000","last_peered":"2023-11-24T06:15:52.788766+0000","last_clean":"2023-11-24T06:15:52.788766+0000","last_became_active":"2023-11-24T06:15:51.804832+0000","last_became_peered":"2023-11-24T06:15:51.804832+0000","last_unstale":"2023-11-24T06:15:52.788766+0000","last_undegraded":"2023-11-24T06:15:52.788766+0000","last_fullsized":"2023-11-24T06:15:52.788766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:21:49.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789431+0000","last_change":"2023-11-24T06:15:51.168020+0000","last_active":"2023-11-24T06:15:52.789431+0000","last_peered":"2023-11-24T06:15:52.789431+0000","last_clean":"2023-11-24T06:15:52.789431+0000","last_became_active":"2023-11-24T06:15:51.167664+0000","last_became_peered":"2023-11-24T06:15:51.167664+0000","last_unstale":"2023-11-24T06:15:52.789431+0000","last_undegraded":"2023-11-24T06:15:52.789431+0000","last_fullsized":"2023-11-24T06:15:52.789431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:17:00.705564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845799+0000","last_change":"2023-11-24T06:15:51.816931+0000","last_active":"2023-11-24T06:15:53.845799+0000","last_peered":"2023-11-24T06:15:53.845799+0000","last_clean":"2023-11-24T06:15:53.845799+0000","last_became_active":"2023-11-24T06:15:51.816611+0000","last_became_peered":"2023-11-24T06:15:51.816611+0000","last_unstale":"2023-11-24T06:15:53.845799+0000","last_undegraded":"2023-11-24T06:15:53.845799+0000","last_fullsized":"2023-11-24T06:15:53.845799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:27.816377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750733+0000","last_change":"2023-11-24T06:15:51.172657+0000","last_active":"2023-11-24T06:15:53.750733+0000","last_peered":"2023-11-24T06:15:53.750733+0000","last_clean":"2023-11-24T06:15:53.750733+0000","last_became_active":"2023-11-24T06:15:51.172152+0000","last_became_peered":"2023-11-24T06:15:51.172152+0000","last_unstale":"2023-11-24T06:15:53.750733+0000","last_undegraded":"2023-11-24T06:15:53.750733+0000","last_fullsized":"2023-11-24T06:15:53.750733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:23:43.714712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.823916+0000","last_change":"2023-11-24T06:15:51.815689+0000","last_active":"2023-11-24T06:15:51.823916+0000","last_peered":"2023-11-24T06:15:51.823916+0000","last_clean":"2023-11-24T06:15:51.823916+0000","last_became_active":"2023-11-24T06:15:51.814742+0000","last_became_peered":"2023-11-24T06:15:51.814742+0000","last_unstale":"2023-11-24T06:15:51.823916+0000","last_undegraded":"2023-11-24T06:15:51.823916+0000","last_fullsized":"2023-11-24T06:15:51.823916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:54:46.540704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183694+0000","last_change":"2023-11-24T06:15:51.164574+0000","last_active":"2023-11-24T06:15:54.183694+0000","last_peered":"2023-11-24T06:15:54.183694+0000","last_clean":"2023-11-24T06:15:54.183694+0000","last_became_active":"2023-11-24T06:15:51.163261+0000","last_became_peered":"2023-11-24T06:15:51.163261+0000","last_unstale":"2023-11-24T06:15:54.183694+0000","last_undegraded":"2023-11-24T06:15:54.183694+0000","last_fullsized":"2023-11-24T06:15:54.183694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:19:30.567064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791508+0000","last_change":"2023-11-24T06:15:51.808104+0000","last_active":"2023-11-24T06:15:52.791508+0000","last_peered":"2023-11-24T06:15:52.791508+0000","last_clean":"2023-11-24T06:15:52.791508+0000","last_became_active":"2023-11-24T06:15:51.807649+0000","last_became_peered":"2023-11-24T06:15:51.807649+0000","last_unstale":"2023-11-24T06:15:52.791508+0000","last_undegraded":"2023-11-24T06:15:52.791508+0000","last_fullsized":"2023-11-24T06:15:52.791508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:47.469041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788424+0000","last_change":"2023-11-24T06:15:51.160561+0000","last_active":"2023-11-24T06:15:52.788424+0000","last_peered":"2023-11-24T06:15:52.788424+0000","last_clean":"2023-11-24T06:15:52.788424+0000","last_became_active":"2023-11-24T06:15:51.160266+0000","last_became_peered":"2023-11-24T06:15:51.160266+0000","last_unstale":"2023-11-24T06:15:52.788424+0000","last_undegraded":"2023-11-24T06:15:52.788424+0000","last_fullsized":"2023-11-24T06:15:52.788424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:09:38.300549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845022+0000","last_change":"2023-11-24T06:15:51.806779+0000","last_active":"2023-11-24T06:15:53.845022+0000","last_peered":"2023-11-24T06:15:53.845022+0000","last_clean":"2023-11-24T06:15:53.845022+0000","last_became_active":"2023-11-24T06:15:51.806429+0000","last_became_peered":"2023-11-24T06:15:51.806429+0000","last_unstale":"2023-11-24T06:15:53.845022+0000","last_undegraded":"2023-11-24T06:15:53.845022+0000","last_fullsized":"2023-11-24T06:15:53.845022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:01.976941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788974+0000","last_change":"2023-11-24T06:15:51.157354+0000","last_active":"2023-11-24T06:15:52.788974+0000","last_peered":"2023-11-24T06:15:52.788974+0000","last_clean":"2023-11-24T06:15:52.788974+0000","last_became_active":"2023-11-24T06:15:51.156981+0000","last_became_peered":"2023-11-24T06:15:51.156981+0000","last_unstale":"2023-11-24T06:15:52.788974+0000","last_undegraded":"2023-11-24T06:15:52.788974+0000","last_fullsized":"2023-11-24T06:15:52.788974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:19:30.567064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845730+0000","last_change":"2023-11-24T06:15:51.823678+0000","last_active":"2023-11-24T06:15:53.845730+0000","last_peered":"2023-11-24T06:15:53.845730+0000","last_clean":"2023-11-24T06:15:53.845730+0000","last_became_active":"2023-11-24T06:15:51.823372+0000","last_became_peered":"2023-11-24T06:15:51.823372+0000","last_unstale":"2023-11-24T06:15:53.845730+0000","last_undegraded":"2023-11-24T06:15:53.845730+0000","last_fullsized":"2023-11-24T06:15:53.845730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:46.484813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184891+0000","last_change":"2023-11-24T06:15:51.167493+0000","last_active":"2023-11-24T06:15:54.184891+0000","last_peered":"2023-11-24T06:15:54.184891+0000","last_clean":"2023-11-24T06:15:54.184891+0000","last_became_active":"2023-11-24T06:15:51.167110+0000","last_became_peered":"2023-11-24T06:15:51.167110+0000","last_unstale":"2023-11-24T06:15:54.184891+0000","last_undegraded":"2023-11-24T06:15:54.184891+0000","last_fullsized":"2023-11-24T06:15:54.184891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:59.444337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749958+0000","last_change":"2023-11-24T06:15:51.816423+0000","last_active":"2023-11-24T06:15:53.749958+0000","last_peered":"2023-11-24T06:15:53.749958+0000","last_clean":"2023-11-24T06:15:53.749958+0000","last_became_active":"2023-11-24T06:15:51.816122+0000","last_became_peered":"2023-11-24T06:15:51.816122+0000","last_unstale":"2023-11-24T06:15:53.749958+0000","last_undegraded":"2023-11-24T06:15:53.749958+0000","last_fullsized":"2023-11-24T06:15:53.749958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:50.460127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793429+0000","last_change":"2023-11-24T06:15:51.803878+0000","last_active":"2023-11-24T06:15:52.793429+0000","last_peered":"2023-11-24T06:15:52.793429+0000","last_clean":"2023-11-24T06:15:52.793429+0000","last_became_active":"2023-11-24T06:15:51.803472+0000","last_became_peered":"2023-11-24T06:15:51.803472+0000","last_unstale":"2023-11-24T06:15:52.793429+0000","last_undegraded":"2023-11-24T06:15:52.793429+0000","last_fullsized":"2023-11-24T06:15:52.793429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:23.431671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184531+0000","last_change":"2023-11-24T06:15:51.177433+0000","last_active":"2023-11-24T06:15:54.184531+0000","last_peered":"2023-11-24T06:15:54.184531+0000","last_clean":"2023-11-24T06:15:54.184531+0000","last_became_active":"2023-11-24T06:15:51.176987+0000","last_became_peered":"2023-11-24T06:15:51.176987+0000","last_unstale":"2023-11-24T06:15:54.184531+0000","last_undegraded":"2023-11-24T06:15:54.184531+0000","last_fullsized":"2023-11-24T06:15:54.184531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:18.981340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793828+0000","last_change":"2023-11-24T06:15:51.815123+0000","last_active":"2023-11-24T06:15:52.793828+0000","last_peered":"2023-11-24T06:15:52.793828+0000","last_clean":"2023-11-24T06:15:52.793828+0000","last_became_active":"2023-11-24T06:15:51.814778+0000","last_became_peered":"2023-11-24T06:15:51.814778+0000","last_unstale":"2023-11-24T06:15:52.793828+0000","last_undegraded":"2023-11-24T06:15:52.793828+0000","last_fullsized":"2023-11-24T06:15:52.793828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:12.167390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185069+0000","last_change":"2023-11-24T06:15:51.169354+0000","last_active":"2023-11-24T06:15:54.185069+0000","last_peered":"2023-11-24T06:15:54.185069+0000","last_clean":"2023-11-24T06:15:54.185069+0000","last_became_active":"2023-11-24T06:15:51.168916+0000","last_became_peered":"2023-11-24T06:15:51.168916+0000","last_unstale":"2023-11-24T06:15:54.185069+0000","last_undegraded":"2023-11-24T06:15:54.185069+0000","last_fullsized":"2023-11-24T06:15:54.185069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:15.261716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.824464+0000","last_change":"2023-11-24T06:15:51.818739+0000","last_active":"2023-11-24T06:15:51.824464+0000","last_peered":"2023-11-24T06:15:51.824464+0000","last_clean":"2023-11-24T06:15:51.824464+0000","last_became_active":"2023-11-24T06:15:51.817316+0000","last_became_peered":"2023-11-24T06:15:51.817316+0000","last_unstale":"2023-11-24T06:15:51.824464+0000","last_undegraded":"2023-11-24T06:15:51.824464+0000","last_fullsized":"2023-11-24T06:15:51.824464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:00:56.411126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845707+0000","last_change":"2023-11-24T06:15:51.152517+0000","last_active":"2023-11-24T06:15:53.845707+0000","last_peered":"2023-11-24T06:15:53.845707+0000","last_clean":"2023-11-24T06:15:53.845707+0000","last_became_active":"2023-11-24T06:15:51.152218+0000","last_became_peered":"2023-11-24T06:15:51.152218+0000","last_unstale":"2023-11-24T06:15:53.845707+0000","last_undegraded":"2023-11-24T06:15:53.845707+0000","last_fullsized":"2023-11-24T06:15:53.845707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:59.444337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845977+0000","last_change":"2023-11-24T06:15:51.817856+0000","last_active":"2023-11-24T06:15:53.845977+0000","last_peered":"2023-11-24T06:15:53.845977+0000","last_clean":"2023-11-24T06:15:53.845977+0000","last_became_active":"2023-11-24T06:15:51.817558+0000","last_became_peered":"2023-11-24T06:15:51.817558+0000","last_unstale":"2023-11-24T06:15:53.845977+0000","last_undegraded":"2023-11-24T06:15:53.845977+0000","last_fullsized":"2023-11-24T06:15:53.845977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:47.291087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.158748+0000","last_change":"2023-11-24T06:15:51.157608+0000","last_active":"2023-11-24T06:15:52.158748+0000","last_peered":"2023-11-24T06:15:52.158748+0000","last_clean":"2023-11-24T06:15:52.158748+0000","last_became_active":"2023-11-24T06:15:51.157330+0000","last_became_peered":"2023-11-24T06:15:51.157330+0000","last_unstale":"2023-11-24T06:15:52.158748+0000","last_undegraded":"2023-11-24T06:15:52.158748+0000","last_fullsized":"2023-11-24T06:15:52.158748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:39:05.215342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":78220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.738}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6879999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":73406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7549999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1230000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5179999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1819999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":78184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.425}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":221184,"data_stored":73406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9930000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.23}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":78094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":78254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.397}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7010000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0219999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.903}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":811008,"data_stored":662081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:15:56.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:15:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.863+0000 7f8b47459640 1 Processor -- start 2023-11-24T06:15:56.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.863+0000 7f8b47459640 1 -- start start 2023-11-24T06:15:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.865+0000 7f8b47459640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8b400d14a0 0x7f8b400d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.865+0000 7f8b47459640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b400b0640 0x7f8b400b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.865+0000 7f8b47459640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b400cf0c0 con 0x7f8b4015d5b0 2023-11-24T06:15:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.865+0000 7f8b47459640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b400b01b0 con 0x7f8b400d14a0 2023-11-24T06:15:56.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.865+0000 7f8b47459640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4015d990 con 0x7f8b400b0640 2023-11-24T06:15:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.865+0000 7f8b45c56640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8b400d14a0 0x7f8b400d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.865+0000 7f8b46457640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8b4015d5b0 0x7f8b400d3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56448/0 (socket says 172.21.15.5:56448) 2023-11-24T06:15:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.865+0000 7f8b46457640 1 -- 172.21.15.5:0/2289918595 learned_addr learned my addr 172.21.15.5:0/2289918595 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.866+0000 7f8b46c58640 1 --2- 172.21.15.5:0/2289918595 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b400b0640 0x7f8b400b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.866+0000 7f8b45c56640 1 -- 172.21.15.5:0/2289918595 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b400b0640 msgr2=0x7f8b400b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.866+0000 7f8b45c56640 1 --2- 172.21.15.5:0/2289918595 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b400b0640 0x7f8b400b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.866+0000 7f8b45c56640 1 -- 172.21.15.5:0/2289918595 >> v1:172.21.15.5:6789/0 conn(0x7f8b4015d5b0 legacy=0x7f8b400d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.866+0000 7f8b45c56640 1 -- 172.21.15.5:0/2289918595 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b400d1e40 con 0x7f8b400d14a0 2023-11-24T06:15:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.866+0000 7f8b45455640 1 -- 172.21.15.5:0/2289918595 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895814658 0 0) 0x7f8b400cf0c0 con 0x7f8b4015d5b0 2023-11-24T06:15:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.867+0000 7f8b45c56640 1 --2- 172.21.15.5:0/2289918595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b400d14a0 0x7f8b400d1900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8b300097a0 tx=0x7f8b3002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e383ed2046a0c8a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.867+0000 7f8b45455640 1 -- 172.21.15.5:0/2289918595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30031830 con 0x7f8b400d14a0 2023-11-24T06:15:56.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.867+0000 7f8b45455640 1 -- 172.21.15.5:0/2289918595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b30031990 con 0x7f8b400d14a0 2023-11-24T06:15:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.867+0000 7f8b45455640 1 -- 172.21.15.5:0/2289918595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30031d20 con 0x7f8b400d14a0 2023-11-24T06:15:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.868+0000 7f8b47459640 1 -- 172.21.15.5:0/2289918595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b400d14a0 msgr2=0x7f8b400d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:56.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.868+0000 7f8b47459640 1 --2- 172.21.15.5:0/2289918595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b400d14a0 0x7f8b400d1900 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8b300097a0 tx=0x7f8b3002b8d0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.868+0000 7f8b47459640 1 -- 172.21.15.5:0/2289918595 shutdown_connections 2023-11-24T06:15:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.868+0000 7f8b47459640 1 --2- 172.21.15.5:0/2289918595 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b400b0640 0x7f8b400b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.868+0000 7f8b47459640 1 --2- 172.21.15.5:0/2289918595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b400d14a0 0x7f8b400d1900 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.868+0000 7f8b47459640 1 -- 172.21.15.5:0/2289918595 >> 172.21.15.5:0/2289918595 conn(0x7f8b4009c3b0 msgr2=0x7f8b400d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.868+0000 7f8b47459640 1 -- 172.21.15.5:0/2289918595 shutdown_connections 2023-11-24T06:15:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.868+0000 7f8b47459640 1 -- 172.21.15.5:0/2289918595 wait complete. 2023-11-24T06:15:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.870+0000 7f8b47459640 1 Processor -- start 2023-11-24T06:15:56.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.870+0000 7f8b47459640 1 -- start start 2023-11-24T06:15:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.871+0000 7f8b47459640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b400b0640 0x7f8b401523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.871+0000 7f8b47459640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b400d14a0 0x7f8b400b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.871+0000 7f8b47459640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4015d5b0 0x7f8b400b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.871+0000 7f8b47459640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400cc620 con 0x7f8b4015d5b0 2023-11-24T06:15:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.871+0000 7f8b47459640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4015dc90 con 0x7f8b400d14a0 2023-11-24T06:15:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.871+0000 7f8b47459640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4015df70 con 0x7f8b400b0640 2023-11-24T06:15:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.872+0000 7f8b46c58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4015d5b0 0x7f8b400b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.872+0000 7f8b46c58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4015d5b0 0x7f8b400b6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35624/0 (socket says 172.21.15.5:35624) 2023-11-24T06:15:56.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.872+0000 7f8b46c58640 1 -- 172.21.15.5:0/3095153092 learned_addr learned my addr 172.21.15.5:0/3095153092 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.872+0000 7f8b46457640 1 --2- 172.21.15.5:0/3095153092 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b400b0640 0x7f8b401523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.872+0000 7f8b45c56640 1 --2- 172.21.15.5:0/3095153092 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b400d14a0 0x7f8b400b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.872+0000 7f8b46c58640 1 -- 172.21.15.5:0/3095153092 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b400b0640 msgr2=0x7f8b401523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.872+0000 7f8b46c58640 1 --2- 172.21.15.5:0/3095153092 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b400b0640 0x7f8b401523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.872+0000 7f8b46c58640 1 -- 172.21.15.5:0/3095153092 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b400d14a0 msgr2=0x7f8b400b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.872+0000 7f8b46c58640 1 --2- 172.21.15.5:0/3095153092 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b400d14a0 0x7f8b400b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:56.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.872+0000 7f8b46c58640 1 -- 172.21.15.5:0/3095153092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b30009400 con 0x7f8b4015d5b0 2023-11-24T06:15:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.872+0000 7f8b46c58640 1 --2- 172.21.15.5:0/3095153092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4015d5b0 0x7f8b400b6180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8b38007e60 tx=0x7f8b3800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.873+0000 7f8b36ffd640 1 -- 172.21.15.5:0/3095153092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38014420 con 0x7f8b4015d5b0 2023-11-24T06:15:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.873+0000 7f8b36ffd640 1 -- 172.21.15.5:0/3095153092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b380145a0 con 0x7f8b4015d5b0 2023-11-24T06:15:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.873+0000 7f8b47459640 1 -- 172.21.15.5:0/3095153092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4015e250 con 0x7f8b4015d5b0 2023-11-24T06:15:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.873+0000 7f8b36ffd640 1 -- 172.21.15.5:0/3095153092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38014950 con 0x7f8b4015d5b0 2023-11-24T06:15:56.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.873+0000 7f8b47459640 1 -- 172.21.15.5:0/3095153092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b400b83a0 con 0x7f8b4015d5b0 2023-11-24T06:15:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.874+0000 7f8b47459640 1 -- 172.21.15.5:0/3095153092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b400cedc0 con 0x7f8b4015d5b0 2023-11-24T06:15:56.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.875+0000 7f8b36ffd640 1 -- 172.21.15.5:0/3095153092 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8b38014ab0 con 0x7f8b4015d5b0 2023-11-24T06:15:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.876+0000 7f8b36ffd640 1 --2- 172.21.15.5:0/3095153092 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8b2406ae40 0x7f8b2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.876+0000 7f8b36ffd640 1 -- 172.21.15.5:0/3095153092 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b38089f70 con 0x7f8b4015d5b0 2023-11-24T06:15:56.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.876+0000 7f8b46457640 1 --2- 172.21.15.5:0/3095153092 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8b2406ae40 0x7f8b2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:56.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.877+0000 7f8b46457640 1 --2- 172.21.15.5:0/3095153092 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8b2406ae40 0x7f8b2406d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c000f60 tx=0x7f8b3c007b70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:56.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:56.881+0000 7f8b36ffd640 1 -- 172.21.15.5:0/3095153092 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b400cedc0 con 0x7f8b4015d5b0 2023-11-24T06:15:57.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.062+0000 7f8b47459640 1 -- 172.21.15.5:0/3095153092 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b4008d0b0 con 0x7f8b2406ae40 2023-11-24T06:15:57.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.083+0000 7f8b36ffd640 1 -- 172.21.15.5:0/3095153092 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264440 (secure 0 0 0) 0x7f8b4008d0b0 con 0x7f8b2406ae40 2023-11-24T06:15:57.088 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:15:57.091 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-11-24T06:15:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.089+0000 7f8b47459640 1 -- 172.21.15.5:0/3095153092 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8b2406ae40 msgr2=0x7f8b2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.089+0000 7f8b47459640 1 --2- 172.21.15.5:0/3095153092 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8b2406ae40 0x7f8b2406d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c000f60 tx=0x7f8b3c007b70 comp rx=0 tx=0).stop 2023-11-24T06:15:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.089+0000 7f8b47459640 1 -- 172.21.15.5:0/3095153092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4015d5b0 msgr2=0x7f8b400b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.089+0000 7f8b47459640 1 --2- 172.21.15.5:0/3095153092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4015d5b0 0x7f8b400b6180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8b38007e60 tx=0x7f8b3800e4b0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.089+0000 7f8b47459640 1 -- 172.21.15.5:0/3095153092 shutdown_connections 2023-11-24T06:15:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.089+0000 7f8b47459640 1 --2- 172.21.15.5:0/3095153092 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8b2406ae40 0x7f8b2406d300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.089+0000 7f8b47459640 1 --2- 172.21.15.5:0/3095153092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b4015d5b0 0x7f8b400b6180 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.089+0000 7f8b47459640 1 --2- 172.21.15.5:0/3095153092 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8b400d14a0 0x7f8b400b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.089+0000 7f8b47459640 1 --2- 172.21.15.5:0/3095153092 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8b400b0640 0x7f8b401523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.089+0000 7f8b47459640 1 -- 172.21.15.5:0/3095153092 >> 172.21.15.5:0/3095153092 conn(0x7f8b4009c3b0 msgr2=0x7f8b400cff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:57.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.090+0000 7f8b47459640 1 -- 172.21.15.5:0/3095153092 shutdown_connections 2023-11-24T06:15:57.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.090+0000 7f8b47459640 1 -- 172.21.15.5:0/3095153092 wait complete. 2023-11-24T06:15:57.109 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-24T06:15:56.088237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235724,"kb_used_data":2912,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738996,"statfs":{"total":773094113280,"available":772852731904,"internally_reserved":0,"allocated":2981888,"data_stored":1790689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.976976"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185145+0000","last_change":"2023-11-24T06:15:51.826913+0000","last_active":"2023-11-24T06:15:54.185145+0000","last_peered":"2023-11-24T06:15:54.185145+0000","last_clean":"2023-11-24T06:15:54.185145+0000","last_became_active":"2023-11-24T06:15:51.826683+0000","last_became_peered":"2023-11-24T06:15:51.826683+0000","last_unstale":"2023-11-24T06:15:54.185145+0000","last_undegraded":"2023-11-24T06:15:54.185145+0000","last_fullsized":"2023-11-24T06:15:54.185145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:45:59.277130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789469+0000","last_change":"2023-11-24T06:15:51.820105+0000","last_active":"2023-11-24T06:15:52.789469+0000","last_peered":"2023-11-24T06:15:52.789469+0000","last_clean":"2023-11-24T06:15:52.789469+0000","last_became_active":"2023-11-24T06:15:51.819862+0000","last_became_peered":"2023-11-24T06:15:51.819862+0000","last_unstale":"2023-11-24T06:15:52.789469+0000","last_undegraded":"2023-11-24T06:15:52.789469+0000","last_fullsized":"2023-11-24T06:15:52.789469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:58.501546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.824288+0000","last_change":"2023-11-24T06:15:51.817345+0000","last_active":"2023-11-24T06:15:51.824288+0000","last_peered":"2023-11-24T06:15:51.824288+0000","last_clean":"2023-11-24T06:15:51.824288+0000","last_became_active":"2023-11-24T06:15:51.815455+0000","last_became_peered":"2023-11-24T06:15:51.815455+0000","last_unstale":"2023-11-24T06:15:51.824288+0000","last_undegraded":"2023-11-24T06:15:51.824288+0000","last_fullsized":"2023-11-24T06:15:51.824288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:32.279875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793545+0000","last_change":"2023-11-24T06:15:51.811576+0000","last_active":"2023-11-24T06:15:52.793545+0000","last_peered":"2023-11-24T06:15:52.793545+0000","last_clean":"2023-11-24T06:15:52.793545+0000","last_became_active":"2023-11-24T06:15:51.811306+0000","last_became_peered":"2023-11-24T06:15:51.811306+0000","last_unstale":"2023-11-24T06:15:52.793545+0000","last_undegraded":"2023-11-24T06:15:52.793545+0000","last_fullsized":"2023-11-24T06:15:52.793545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:34.521019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184624+0000","last_change":"2023-11-24T06:15:51.821890+0000","last_active":"2023-11-24T06:15:54.184624+0000","last_peered":"2023-11-24T06:15:54.184624+0000","last_clean":"2023-11-24T06:15:54.184624+0000","last_became_active":"2023-11-24T06:15:51.821632+0000","last_became_peered":"2023-11-24T06:15:51.821632+0000","last_unstale":"2023-11-24T06:15:54.184624+0000","last_undegraded":"2023-11-24T06:15:54.184624+0000","last_fullsized":"2023-11-24T06:15:54.184624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:54.384409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792886+0000","last_change":"2023-11-24T06:15:51.828326+0000","last_active":"2023-11-24T06:15:52.792886+0000","last_peered":"2023-11-24T06:15:52.792886+0000","last_clean":"2023-11-24T06:15:52.792886+0000","last_became_active":"2023-11-24T06:15:51.827922+0000","last_became_peered":"2023-11-24T06:15:51.827922+0000","last_unstale":"2023-11-24T06:15:52.792886+0000","last_undegraded":"2023-11-24T06:15:52.792886+0000","last_fullsized":"2023-11-24T06:15:52.792886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:25.825619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183858+0000","last_change":"2023-11-24T06:15:51.825625+0000","last_active":"2023-11-24T06:15:54.183858+0000","last_peered":"2023-11-24T06:15:54.183858+0000","last_clean":"2023-11-24T06:15:54.183858+0000","last_became_active":"2023-11-24T06:15:51.825308+0000","last_became_peered":"2023-11-24T06:15:51.825308+0000","last_unstale":"2023-11-24T06:15:54.183858+0000","last_undegraded":"2023-11-24T06:15:54.183858+0000","last_fullsized":"2023-11-24T06:15:54.183858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:58.501546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844647+0000","last_change":"2023-11-24T06:15:51.817175+0000","last_active":"2023-11-24T06:15:53.844647+0000","last_peered":"2023-11-24T06:15:53.844647+0000","last_clean":"2023-11-24T06:15:53.844647+0000","last_became_active":"2023-11-24T06:15:51.816363+0000","last_became_peered":"2023-11-24T06:15:51.816363+0000","last_unstale":"2023-11-24T06:15:53.844647+0000","last_undegraded":"2023-11-24T06:15:53.844647+0000","last_fullsized":"2023-11-24T06:15:53.844647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:00:51.554978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185142+0000","last_change":"2023-11-24T06:15:51.829604+0000","last_active":"2023-11-24T06:15:54.185142+0000","last_peered":"2023-11-24T06:15:54.185142+0000","last_clean":"2023-11-24T06:15:54.185142+0000","last_became_active":"2023-11-24T06:15:51.829291+0000","last_became_peered":"2023-11-24T06:15:51.829291+0000","last_unstale":"2023-11-24T06:15:54.185142+0000","last_undegraded":"2023-11-24T06:15:54.185142+0000","last_fullsized":"2023-11-24T06:15:54.185142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:57:53.020152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750691+0000","last_change":"2023-11-24T06:15:51.820996+0000","last_active":"2023-11-24T06:15:53.750691+0000","last_peered":"2023-11-24T06:15:53.750691+0000","last_clean":"2023-11-24T06:15:53.750691+0000","last_became_active":"2023-11-24T06:15:51.820018+0000","last_became_peered":"2023-11-24T06:15:51.820018+0000","last_unstale":"2023-11-24T06:15:53.750691+0000","last_undegraded":"2023-11-24T06:15:53.750691+0000","last_fullsized":"2023-11-24T06:15:53.750691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:05.782767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750915+0000","last_change":"2023-11-24T06:15:51.806762+0000","last_active":"2023-11-24T06:15:53.750915+0000","last_peered":"2023-11-24T06:15:53.750915+0000","last_clean":"2023-11-24T06:15:53.750915+0000","last_became_active":"2023-11-24T06:15:51.806476+0000","last_became_peered":"2023-11-24T06:15:51.806476+0000","last_unstale":"2023-11-24T06:15:53.750915+0000","last_undegraded":"2023-11-24T06:15:53.750915+0000","last_fullsized":"2023-11-24T06:15:53.750915+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:25.825619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791243+0000","last_change":"2023-11-24T06:15:51.807959+0000","last_active":"2023-11-24T06:15:52.791243+0000","last_peered":"2023-11-24T06:15:52.791243+0000","last_clean":"2023-11-24T06:15:52.791243+0000","last_became_active":"2023-11-24T06:15:51.807625+0000","last_became_peered":"2023-11-24T06:15:51.807625+0000","last_unstale":"2023-11-24T06:15:52.791243+0000","last_undegraded":"2023-11-24T06:15:52.791243+0000","last_fullsized":"2023-11-24T06:15:52.791243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:37.913398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845276+0000","last_change":"2023-11-24T06:15:51.820393+0000","last_active":"2023-11-24T06:15:53.845276+0000","last_peered":"2023-11-24T06:15:53.845276+0000","last_clean":"2023-11-24T06:15:53.845276+0000","last_became_active":"2023-11-24T06:15:51.820092+0000","last_became_peered":"2023-11-24T06:15:51.820092+0000","last_unstale":"2023-11-24T06:15:53.845276+0000","last_undegraded":"2023-11-24T06:15:53.845276+0000","last_fullsized":"2023-11-24T06:15:53.845276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:16.273773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.824842+0000","last_change":"2023-11-24T06:15:51.822418+0000","last_active":"2023-11-24T06:15:51.824842+0000","last_peered":"2023-11-24T06:15:51.824842+0000","last_clean":"2023-11-24T06:15:51.824842+0000","last_became_active":"2023-11-24T06:15:51.821162+0000","last_became_peered":"2023-11-24T06:15:51.821162+0000","last_unstale":"2023-11-24T06:15:51.824842+0000","last_undegraded":"2023-11-24T06:15:51.824842+0000","last_fullsized":"2023-11-24T06:15:51.824842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:43.354531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788580+0000","last_change":"2023-11-24T06:15:51.808930+0000","last_active":"2023-11-24T06:15:52.788580+0000","last_peered":"2023-11-24T06:15:52.788580+0000","last_clean":"2023-11-24T06:15:52.788580+0000","last_became_active":"2023-11-24T06:15:51.808593+0000","last_became_peered":"2023-11-24T06:15:51.808593+0000","last_unstale":"2023-11-24T06:15:52.788580+0000","last_undegraded":"2023-11-24T06:15:52.788580+0000","last_fullsized":"2023-11-24T06:15:52.788580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:43.291318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792334+0000","last_change":"2023-11-24T06:15:51.829868+0000","last_active":"2023-11-24T06:15:52.792334+0000","last_peered":"2023-11-24T06:15:52.792334+0000","last_clean":"2023-11-24T06:15:52.792334+0000","last_became_active":"2023-11-24T06:15:51.828875+0000","last_became_peered":"2023-11-24T06:15:51.828875+0000","last_unstale":"2023-11-24T06:15:52.792334+0000","last_undegraded":"2023-11-24T06:15:52.792334+0000","last_fullsized":"2023-11-24T06:15:52.792334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:32.225953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.794592+0000","last_change":"2023-11-24T06:15:51.814343+0000","last_active":"2023-11-24T06:15:52.794592+0000","last_peered":"2023-11-24T06:15:52.794592+0000","last_clean":"2023-11-24T06:15:52.794592+0000","last_became_active":"2023-11-24T06:15:51.814003+0000","last_became_peered":"2023-11-24T06:15:51.814003+0000","last_unstale":"2023-11-24T06:15:52.794592+0000","last_undegraded":"2023-11-24T06:15:52.794592+0000","last_fullsized":"2023-11-24T06:15:52.794592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:26:09.975671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.139925+0000","last_change":"2023-11-24T06:15:51.815884+0000","last_active":"2023-11-24T06:15:53.139925+0000","last_peered":"2023-11-24T06:15:53.139925+0000","last_clean":"2023-11-24T06:15:53.139925+0000","last_became_active":"2023-11-24T06:15:51.814549+0000","last_became_peered":"2023-11-24T06:15:51.814549+0000","last_unstale":"2023-11-24T06:15:53.139925+0000","last_undegraded":"2023-11-24T06:15:53.139925+0000","last_fullsized":"2023-11-24T06:15:53.139925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:43.354531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185458+0000","last_change":"2023-11-24T06:15:51.823265+0000","last_active":"2023-11-24T06:15:54.185458+0000","last_peered":"2023-11-24T06:15:54.185458+0000","last_clean":"2023-11-24T06:15:54.185458+0000","last_became_active":"2023-11-24T06:15:51.823026+0000","last_became_peered":"2023-11-24T06:15:51.823026+0000","last_unstale":"2023-11-24T06:15:54.185458+0000","last_undegraded":"2023-11-24T06:15:54.185458+0000","last_fullsized":"2023-11-24T06:15:54.185458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:48.210729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750151+0000","last_change":"2023-11-24T06:15:51.818614+0000","last_active":"2023-11-24T06:15:53.750151+0000","last_peered":"2023-11-24T06:15:53.750151+0000","last_clean":"2023-11-24T06:15:53.750151+0000","last_became_active":"2023-11-24T06:15:51.818250+0000","last_became_peered":"2023-11-24T06:15:51.818250+0000","last_unstale":"2023-11-24T06:15:53.750151+0000","last_undegraded":"2023-11-24T06:15:53.750151+0000","last_fullsized":"2023-11-24T06:15:53.750151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:07.460859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.824083+0000","last_change":"2023-11-24T06:15:51.815798+0000","last_active":"2023-11-24T06:15:51.824083+0000","last_peered":"2023-11-24T06:15:51.824083+0000","last_clean":"2023-11-24T06:15:51.824083+0000","last_became_active":"2023-11-24T06:15:51.815531+0000","last_became_peered":"2023-11-24T06:15:51.815531+0000","last_unstale":"2023-11-24T06:15:51.824083+0000","last_undegraded":"2023-11-24T06:15:51.824083+0000","last_fullsized":"2023-11-24T06:15:51.824083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:08.205306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.794291+0000","last_change":"2023-11-24T06:15:51.813391+0000","last_active":"2023-11-24T06:15:52.794291+0000","last_peered":"2023-11-24T06:15:52.794291+0000","last_clean":"2023-11-24T06:15:52.794291+0000","last_became_active":"2023-11-24T06:15:51.813066+0000","last_became_peered":"2023-11-24T06:15:51.813066+0000","last_unstale":"2023-11-24T06:15:52.794291+0000","last_undegraded":"2023-11-24T06:15:52.794291+0000","last_fullsized":"2023-11-24T06:15:52.794291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:54:56.970614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844721+0000","last_change":"2023-11-24T06:15:51.812820+0000","last_active":"2023-11-24T06:15:53.844721+0000","last_peered":"2023-11-24T06:15:53.844721+0000","last_clean":"2023-11-24T06:15:53.844721+0000","last_became_active":"2023-11-24T06:15:51.812517+0000","last_became_peered":"2023-11-24T06:15:51.812517+0000","last_unstale":"2023-11-24T06:15:53.844721+0000","last_undegraded":"2023-11-24T06:15:53.844721+0000","last_fullsized":"2023-11-24T06:15:53.844721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:25.102989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792262+0000","last_change":"2023-11-24T06:15:51.808797+0000","last_active":"2023-11-24T06:15:52.792262+0000","last_peered":"2023-11-24T06:15:52.792262+0000","last_clean":"2023-11-24T06:15:52.792262+0000","last_became_active":"2023-11-24T06:15:51.808481+0000","last_became_peered":"2023-11-24T06:15:51.808481+0000","last_unstale":"2023-11-24T06:15:52.792262+0000","last_undegraded":"2023-11-24T06:15:52.792262+0000","last_fullsized":"2023-11-24T06:15:52.792262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:05.782767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845642+0000","last_change":"2023-11-24T06:15:51.818331+0000","last_active":"2023-11-24T06:15:53.845642+0000","last_peered":"2023-11-24T06:15:53.845642+0000","last_clean":"2023-11-24T06:15:53.845642+0000","last_became_active":"2023-11-24T06:15:51.818011+0000","last_became_peered":"2023-11-24T06:15:51.818011+0000","last_unstale":"2023-11-24T06:15:53.845642+0000","last_undegraded":"2023-11-24T06:15:53.845642+0000","last_fullsized":"2023-11-24T06:15:53.845642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:58.501546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750817+0000","last_change":"2023-11-24T06:15:51.819933+0000","last_active":"2023-11-24T06:15:53.750817+0000","last_peered":"2023-11-24T06:15:53.750817+0000","last_clean":"2023-11-24T06:15:53.750817+0000","last_became_active":"2023-11-24T06:15:51.819592+0000","last_became_peered":"2023-11-24T06:15:51.819592+0000","last_unstale":"2023-11-24T06:15:53.750817+0000","last_undegraded":"2023-11-24T06:15:53.750817+0000","last_fullsized":"2023-11-24T06:15:53.750817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:32.225953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788974+0000","last_change":"2023-11-24T06:15:51.810396+0000","last_active":"2023-11-24T06:15:52.788974+0000","last_peered":"2023-11-24T06:15:52.788974+0000","last_clean":"2023-11-24T06:15:52.788974+0000","last_became_active":"2023-11-24T06:15:51.809939+0000","last_became_peered":"2023-11-24T06:15:51.809939+0000","last_unstale":"2023-11-24T06:15:52.788974+0000","last_undegraded":"2023-11-24T06:15:52.788974+0000","last_fullsized":"2023-11-24T06:15:52.788974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:16.273773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845858+0000","last_change":"2023-11-24T06:15:51.818877+0000","last_active":"2023-11-24T06:15:53.845858+0000","last_peered":"2023-11-24T06:15:53.845858+0000","last_clean":"2023-11-24T06:15:53.845858+0000","last_became_active":"2023-11-24T06:15:51.818068+0000","last_became_peered":"2023-11-24T06:15:51.818068+0000","last_unstale":"2023-11-24T06:15:53.845858+0000","last_undegraded":"2023-11-24T06:15:53.845858+0000","last_fullsized":"2023-11-24T06:15:53.845858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:54.384409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.825444+0000","last_change":"2023-11-24T06:15:51.821479+0000","last_active":"2023-11-24T06:15:51.825444+0000","last_peered":"2023-11-24T06:15:51.825444+0000","last_clean":"2023-11-24T06:15:51.825444+0000","last_became_active":"2023-11-24T06:15:51.820978+0000","last_became_peered":"2023-11-24T06:15:51.820978+0000","last_unstale":"2023-11-24T06:15:51.825444+0000","last_undegraded":"2023-11-24T06:15:51.825444+0000","last_fullsized":"2023-11-24T06:15:51.825444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:08.846287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793272+0000","last_change":"2023-11-24T06:15:51.808157+0000","last_active":"2023-11-24T06:15:52.793272+0000","last_peered":"2023-11-24T06:15:52.793272+0000","last_clean":"2023-11-24T06:15:52.793272+0000","last_became_active":"2023-11-24T06:15:51.807868+0000","last_became_peered":"2023-11-24T06:15:51.807868+0000","last_unstale":"2023-11-24T06:15:52.793272+0000","last_undegraded":"2023-11-24T06:15:52.793272+0000","last_fullsized":"2023-11-24T06:15:52.793272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:41:05.120915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.140134+0000","last_change":"2023-11-24T06:15:51.809534+0000","last_active":"2023-11-24T06:15:53.140134+0000","last_peered":"2023-11-24T06:15:53.140134+0000","last_clean":"2023-11-24T06:15:53.140134+0000","last_became_active":"2023-11-24T06:15:51.809037+0000","last_became_peered":"2023-11-24T06:15:51.809037+0000","last_unstale":"2023-11-24T06:15:53.140134+0000","last_undegraded":"2023-11-24T06:15:53.140134+0000","last_fullsized":"2023-11-24T06:15:53.140134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:38.032032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750282+0000","last_change":"2023-11-24T06:15:51.825619+0000","last_active":"2023-11-24T06:15:53.750282+0000","last_peered":"2023-11-24T06:15:53.750282+0000","last_clean":"2023-11-24T06:15:53.750282+0000","last_became_active":"2023-11-24T06:15:51.825149+0000","last_became_peered":"2023-11-24T06:15:51.825149+0000","last_unstale":"2023-11-24T06:15:53.750282+0000","last_undegraded":"2023-11-24T06:15:53.750282+0000","last_fullsized":"2023-11-24T06:15:53.750282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:12:12.520019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791884+0000","last_change":"2023-11-24T06:15:51.810514+0000","last_active":"2023-11-24T06:15:52.791884+0000","last_peered":"2023-11-24T06:15:52.791884+0000","last_clean":"2023-11-24T06:15:52.791884+0000","last_became_active":"2023-11-24T06:15:51.807943+0000","last_became_peered":"2023-11-24T06:15:51.807943+0000","last_unstale":"2023-11-24T06:15:52.791884+0000","last_undegraded":"2023-11-24T06:15:52.791884+0000","last_fullsized":"2023-11-24T06:15:52.791884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:19.993397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845626+0000","last_change":"2023-11-24T06:15:51.164865+0000","last_active":"2023-11-24T06:15:53.845626+0000","last_peered":"2023-11-24T06:15:53.845626+0000","last_clean":"2023-11-24T06:15:53.845626+0000","last_became_active":"2023-11-24T06:15:51.164524+0000","last_became_peered":"2023-11-24T06:15:51.164524+0000","last_unstale":"2023-11-24T06:15:53.845626+0000","last_undegraded":"2023-11-24T06:15:53.845626+0000","last_fullsized":"2023-11-24T06:15:53.845626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:02.689172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.825431+0000","last_change":"2023-11-24T06:15:51.818737+0000","last_active":"2023-11-24T06:15:51.825431+0000","last_peered":"2023-11-24T06:15:51.825431+0000","last_clean":"2023-11-24T06:15:51.825431+0000","last_became_active":"2023-11-24T06:15:51.817354+0000","last_became_peered":"2023-11-24T06:15:51.817354+0000","last_unstale":"2023-11-24T06:15:51.825431+0000","last_undegraded":"2023-11-24T06:15:51.825431+0000","last_fullsized":"2023-11-24T06:15:51.825431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:27:22.945202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749981+0000","last_change":"2023-11-24T06:15:51.169166+0000","last_active":"2023-11-24T06:15:53.749981+0000","last_peered":"2023-11-24T06:15:53.749981+0000","last_clean":"2023-11-24T06:15:53.749981+0000","last_became_active":"2023-11-24T06:15:51.168863+0000","last_became_peered":"2023-11-24T06:15:51.168863+0000","last_unstale":"2023-11-24T06:15:53.749981+0000","last_undegraded":"2023-11-24T06:15:53.749981+0000","last_fullsized":"2023-11-24T06:15:53.749981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:16.146869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844773+0000","last_change":"2023-11-24T06:15:51.165957+0000","last_active":"2023-11-24T06:15:53.844773+0000","last_peered":"2023-11-24T06:15:53.844773+0000","last_clean":"2023-11-24T06:15:53.844773+0000","last_became_active":"2023-11-24T06:15:51.165575+0000","last_became_peered":"2023-11-24T06:15:51.165575+0000","last_unstale":"2023-11-24T06:15:53.844773+0000","last_undegraded":"2023-11-24T06:15:53.844773+0000","last_fullsized":"2023-11-24T06:15:53.844773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:09.461899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844747+0000","last_change":"2023-11-24T06:15:51.818508+0000","last_active":"2023-11-24T06:15:53.844747+0000","last_peered":"2023-11-24T06:15:53.844747+0000","last_clean":"2023-11-24T06:15:53.844747+0000","last_became_active":"2023-11-24T06:15:51.818158+0000","last_became_peered":"2023-11-24T06:15:51.818158+0000","last_unstale":"2023-11-24T06:15:53.844747+0000","last_undegraded":"2023-11-24T06:15:53.844747+0000","last_fullsized":"2023-11-24T06:15:53.844747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:43.291318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792534+0000","last_change":"2023-11-24T06:15:51.168437+0000","last_active":"2023-11-24T06:15:52.792534+0000","last_peered":"2023-11-24T06:15:52.792534+0000","last_clean":"2023-11-24T06:15:52.792534+0000","last_became_active":"2023-11-24T06:15:51.162079+0000","last_became_peered":"2023-11-24T06:15:51.162079+0000","last_unstale":"2023-11-24T06:15:52.792534+0000","last_undegraded":"2023-11-24T06:15:52.792534+0000","last_fullsized":"2023-11-24T06:15:52.792534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:07.834230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788318+0000","last_change":"2023-11-24T06:15:51.811023+0000","last_active":"2023-11-24T06:15:52.788318+0000","last_peered":"2023-11-24T06:15:52.788318+0000","last_clean":"2023-11-24T06:15:52.788318+0000","last_became_active":"2023-11-24T06:15:51.810368+0000","last_became_peered":"2023-11-24T06:15:51.810368+0000","last_unstale":"2023-11-24T06:15:52.788318+0000","last_undegraded":"2023-11-24T06:15:52.788318+0000","last_fullsized":"2023-11-24T06:15:52.788318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:25.102989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750531+0000","last_change":"2023-11-24T06:15:51.175246+0000","last_active":"2023-11-24T06:15:53.750531+0000","last_peered":"2023-11-24T06:15:53.750531+0000","last_clean":"2023-11-24T06:15:53.750531+0000","last_became_active":"2023-11-24T06:15:51.174941+0000","last_became_peered":"2023-11-24T06:15:51.174941+0000","last_unstale":"2023-11-24T06:15:53.750531+0000","last_undegraded":"2023-11-24T06:15:53.750531+0000","last_fullsized":"2023-11-24T06:15:53.750531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:48.317425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793290+0000","last_change":"2023-11-24T06:15:51.822074+0000","last_active":"2023-11-24T06:15:52.793290+0000","last_peered":"2023-11-24T06:15:52.793290+0000","last_clean":"2023-11-24T06:15:52.793290+0000","last_became_active":"2023-11-24T06:15:51.821749+0000","last_became_peered":"2023-11-24T06:15:51.821749+0000","last_unstale":"2023-11-24T06:15:52.793290+0000","last_undegraded":"2023-11-24T06:15:52.793290+0000","last_fullsized":"2023-11-24T06:15:52.793290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:47:30.989438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793436+0000","last_change":"2023-11-24T06:15:51.175128+0000","last_active":"2023-11-24T06:15:52.793436+0000","last_peered":"2023-11-24T06:15:52.793436+0000","last_clean":"2023-11-24T06:15:52.793436+0000","last_became_active":"2023-11-24T06:15:51.173422+0000","last_became_peered":"2023-11-24T06:15:51.173422+0000","last_unstale":"2023-11-24T06:15:52.793436+0000","last_undegraded":"2023-11-24T06:15:52.793436+0000","last_fullsized":"2023-11-24T06:15:52.793436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:12:11.507962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.822454+0000","last_change":"2023-11-24T06:15:51.814210+0000","last_active":"2023-11-24T06:15:51.822454+0000","last_peered":"2023-11-24T06:15:51.822454+0000","last_clean":"2023-11-24T06:15:51.822454+0000","last_became_active":"2023-11-24T06:15:51.813951+0000","last_became_peered":"2023-11-24T06:15:51.813951+0000","last_unstale":"2023-11-24T06:15:51.822454+0000","last_undegraded":"2023-11-24T06:15:51.822454+0000","last_fullsized":"2023-11-24T06:15:51.822454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:48.919036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789082+0000","last_change":"2023-11-24T06:15:51.176266+0000","last_active":"2023-11-24T06:15:52.789082+0000","last_peered":"2023-11-24T06:15:52.789082+0000","last_clean":"2023-11-24T06:15:52.789082+0000","last_became_active":"2023-11-24T06:15:51.175918+0000","last_became_peered":"2023-11-24T06:15:51.175918+0000","last_unstale":"2023-11-24T06:15:52.789082+0000","last_undegraded":"2023-11-24T06:15:52.789082+0000","last_fullsized":"2023-11-24T06:15:52.789082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:18.981340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791650+0000","last_change":"2023-11-24T06:15:51.819556+0000","last_active":"2023-11-24T06:15:52.791650+0000","last_peered":"2023-11-24T06:15:52.791650+0000","last_clean":"2023-11-24T06:15:52.791650+0000","last_became_active":"2023-11-24T06:15:51.818398+0000","last_became_peered":"2023-11-24T06:15:51.818398+0000","last_unstale":"2023-11-24T06:15:52.791650+0000","last_undegraded":"2023-11-24T06:15:52.791650+0000","last_fullsized":"2023-11-24T06:15:52.791650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:41.065020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750080+0000","last_change":"2023-11-24T06:15:51.169595+0000","last_active":"2023-11-24T06:15:53.750080+0000","last_peered":"2023-11-24T06:15:53.750080+0000","last_clean":"2023-11-24T06:15:53.750080+0000","last_became_active":"2023-11-24T06:15:51.167755+0000","last_became_peered":"2023-11-24T06:15:51.167755+0000","last_unstale":"2023-11-24T06:15:53.750080+0000","last_undegraded":"2023-11-24T06:15:53.750080+0000","last_fullsized":"2023-11-24T06:15:53.750080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:37.019975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184774+0000","last_change":"2023-11-24T06:15:51.821305+0000","last_active":"2023-11-24T06:15:54.184774+0000","last_peered":"2023-11-24T06:15:54.184774+0000","last_clean":"2023-11-24T06:15:54.184774+0000","last_became_active":"2023-11-24T06:15:51.821061+0000","last_became_peered":"2023-11-24T06:15:51.821061+0000","last_unstale":"2023-11-24T06:15:54.184774+0000","last_undegraded":"2023-11-24T06:15:54.184774+0000","last_fullsized":"2023-11-24T06:15:54.184774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:24.298719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.158012+0000","last_change":"2023-11-24T06:15:51.150730+0000","last_active":"2023-11-24T06:15:52.158012+0000","last_peered":"2023-11-24T06:15:52.158012+0000","last_clean":"2023-11-24T06:15:52.158012+0000","last_became_active":"2023-11-24T06:15:51.147375+0000","last_became_peered":"2023-11-24T06:15:51.147375+0000","last_unstale":"2023-11-24T06:15:52.158012+0000","last_undegraded":"2023-11-24T06:15:52.158012+0000","last_fullsized":"2023-11-24T06:15:52.158012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:24:38.346475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.824860+0000","last_change":"2023-11-24T06:15:51.822499+0000","last_active":"2023-11-24T06:15:51.824860+0000","last_peered":"2023-11-24T06:15:51.824860+0000","last_clean":"2023-11-24T06:15:51.824860+0000","last_became_active":"2023-11-24T06:15:51.822192+0000","last_became_peered":"2023-11-24T06:15:51.822192+0000","last_unstale":"2023-11-24T06:15:51.824860+0000","last_undegraded":"2023-11-24T06:15:51.824860+0000","last_fullsized":"2023-11-24T06:15:51.824860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:47:59.130691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793561+0000","last_change":"2023-11-24T06:15:51.174234+0000","last_active":"2023-11-24T06:15:52.793561+0000","last_peered":"2023-11-24T06:15:52.793561+0000","last_clean":"2023-11-24T06:15:52.793561+0000","last_became_active":"2023-11-24T06:15:51.173741+0000","last_became_peered":"2023-11-24T06:15:51.173741+0000","last_unstale":"2023-11-24T06:15:52.793561+0000","last_undegraded":"2023-11-24T06:15:52.793561+0000","last_fullsized":"2023-11-24T06:15:52.793561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:16.146869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184654+0000","last_change":"2023-11-24T06:15:51.828660+0000","last_active":"2023-11-24T06:15:54.184654+0000","last_peered":"2023-11-24T06:15:54.184654+0000","last_clean":"2023-11-24T06:15:54.184654+0000","last_became_active":"2023-11-24T06:15:51.828176+0000","last_became_peered":"2023-11-24T06:15:51.828176+0000","last_unstale":"2023-11-24T06:15:54.184654+0000","last_undegraded":"2023-11-24T06:15:54.184654+0000","last_fullsized":"2023-11-24T06:15:54.184654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:47.291087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749753+0000","last_change":"2023-11-24T06:15:51.171268+0000","last_active":"2023-11-24T06:15:53.749753+0000","last_peered":"2023-11-24T06:15:53.749753+0000","last_clean":"2023-11-24T06:15:53.749753+0000","last_became_active":"2023-11-24T06:15:51.170887+0000","last_became_peered":"2023-11-24T06:15:51.170887+0000","last_unstale":"2023-11-24T06:15:53.749753+0000","last_undegraded":"2023-11-24T06:15:53.749753+0000","last_fullsized":"2023-11-24T06:15:53.749753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:24:38.346475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749967+0000","last_change":"2023-11-24T06:15:51.825705+0000","last_active":"2023-11-24T06:15:53.749967+0000","last_peered":"2023-11-24T06:15:53.749967+0000","last_clean":"2023-11-24T06:15:53.749967+0000","last_became_active":"2023-11-24T06:15:51.825372+0000","last_became_peered":"2023-11-24T06:15:51.825372+0000","last_unstale":"2023-11-24T06:15:53.749967+0000","last_undegraded":"2023-11-24T06:15:53.749967+0000","last_fullsized":"2023-11-24T06:15:53.749967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:57:57.197250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.139651+0000","last_change":"2023-11-24T06:15:51.155610+0000","last_active":"2023-11-24T06:15:53.139651+0000","last_peered":"2023-11-24T06:15:53.139651+0000","last_clean":"2023-11-24T06:15:53.139651+0000","last_became_active":"2023-11-24T06:15:51.155318+0000","last_became_peered":"2023-11-24T06:15:51.155318+0000","last_unstale":"2023-11-24T06:15:53.139651+0000","last_undegraded":"2023-11-24T06:15:53.139651+0000","last_fullsized":"2023-11-24T06:15:53.139651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:07.193249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183869+0000","last_change":"2023-11-24T06:15:51.825462+0000","last_active":"2023-11-24T06:15:54.183869+0000","last_peered":"2023-11-24T06:15:54.183869+0000","last_clean":"2023-11-24T06:15:54.183869+0000","last_became_active":"2023-11-24T06:15:51.825148+0000","last_became_peered":"2023-11-24T06:15:51.825148+0000","last_unstale":"2023-11-24T06:15:54.183869+0000","last_undegraded":"2023-11-24T06:15:54.183869+0000","last_fullsized":"2023-11-24T06:15:54.183869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:46.484813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.158814+0000","last_change":"2023-11-24T06:15:51.151498+0000","last_active":"2023-11-24T06:15:52.158814+0000","last_peered":"2023-11-24T06:15:52.158814+0000","last_clean":"2023-11-24T06:15:52.158814+0000","last_became_active":"2023-11-24T06:15:51.151116+0000","last_became_peered":"2023-11-24T06:15:51.151116+0000","last_unstale":"2023-11-24T06:15:52.158814+0000","last_undegraded":"2023-11-24T06:15:52.158814+0000","last_fullsized":"2023-11-24T06:15:52.158814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:32.153844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184490+0000","last_change":"2023-11-24T06:15:51.826667+0000","last_active":"2023-11-24T06:15:54.184490+0000","last_peered":"2023-11-24T06:15:54.184490+0000","last_clean":"2023-11-24T06:15:54.184490+0000","last_became_active":"2023-11-24T06:15:51.826286+0000","last_became_peered":"2023-11-24T06:15:51.826286+0000","last_unstale":"2023-11-24T06:15:54.184490+0000","last_undegraded":"2023-11-24T06:15:54.184490+0000","last_fullsized":"2023-11-24T06:15:54.184490+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:27.816377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.140108+0000","last_change":"2023-11-24T06:15:51.156776+0000","last_active":"2023-11-24T06:15:53.140108+0000","last_peered":"2023-11-24T06:15:53.140108+0000","last_clean":"2023-11-24T06:15:53.140108+0000","last_became_active":"2023-11-24T06:15:51.156313+0000","last_became_peered":"2023-11-24T06:15:51.156313+0000","last_unstale":"2023-11-24T06:15:53.140108+0000","last_undegraded":"2023-11-24T06:15:53.140108+0000","last_fullsized":"2023-11-24T06:15:53.140108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:47:58.118634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185599+0000","last_change":"2023-11-24T06:15:51.825443+0000","last_active":"2023-11-24T06:15:54.185599+0000","last_peered":"2023-11-24T06:15:54.185599+0000","last_clean":"2023-11-24T06:15:54.185599+0000","last_became_active":"2023-11-24T06:15:51.825149+0000","last_became_peered":"2023-11-24T06:15:51.825149+0000","last_unstale":"2023-11-24T06:15:54.185599+0000","last_undegraded":"2023-11-24T06:15:54.185599+0000","last_fullsized":"2023-11-24T06:15:54.185599+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:37.832748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789200+0000","last_change":"2023-11-24T06:15:51.158822+0000","last_active":"2023-11-24T06:15:52.789200+0000","last_peered":"2023-11-24T06:15:52.789200+0000","last_clean":"2023-11-24T06:15:52.789200+0000","last_became_active":"2023-11-24T06:15:51.158544+0000","last_became_peered":"2023-11-24T06:15:51.158544+0000","last_unstale":"2023-11-24T06:15:52.789200+0000","last_undegraded":"2023-11-24T06:15:52.789200+0000","last_fullsized":"2023-11-24T06:15:52.789200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:02.689172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.794289+0000","last_change":"2023-11-24T06:15:51.808175+0000","last_active":"2023-11-24T06:15:52.794289+0000","last_peered":"2023-11-24T06:15:52.794289+0000","last_clean":"2023-11-24T06:15:52.794289+0000","last_became_active":"2023-11-24T06:15:51.807904+0000","last_became_peered":"2023-11-24T06:15:51.807904+0000","last_unstale":"2023-11-24T06:15:52.794289+0000","last_undegraded":"2023-11-24T06:15:52.794289+0000","last_fullsized":"2023-11-24T06:15:52.794289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:54.525459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.139914+0000","last_change":"2023-11-24T06:15:51.175398+0000","last_active":"2023-11-24T06:15:53.139914+0000","last_peered":"2023-11-24T06:15:53.139914+0000","last_clean":"2023-11-24T06:15:53.139914+0000","last_became_active":"2023-11-24T06:15:51.175039+0000","last_became_peered":"2023-11-24T06:15:51.175039+0000","last_unstale":"2023-11-24T06:15:53.139914+0000","last_undegraded":"2023-11-24T06:15:53.139914+0000","last_fullsized":"2023-11-24T06:15:53.139914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:07.834230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185074+0000","last_change":"2023-11-24T06:15:51.829803+0000","last_active":"2023-11-24T06:15:54.185074+0000","last_peered":"2023-11-24T06:15:54.185074+0000","last_clean":"2023-11-24T06:15:54.185074+0000","last_became_active":"2023-11-24T06:15:51.829417+0000","last_became_peered":"2023-11-24T06:15:51.829417+0000","last_unstale":"2023-11-24T06:15:54.185074+0000","last_undegraded":"2023-11-24T06:15:54.185074+0000","last_fullsized":"2023-11-24T06:15:54.185074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:55:04.397371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.810683+0000","last_change":"2023-11-24T06:15:42.021608+0000","last_active":"2023-11-24T06:15:51.810683+0000","last_peered":"2023-11-24T06:15:51.810683+0000","last_clean":"2023-11-24T06:15:51.810683+0000","last_became_active":"2023-11-24T06:15:42.018847+0000","last_became_peered":"2023-11-24T06:15:42.018847+0000","last_unstale":"2023-11-24T06:15:51.810683+0000","last_undegraded":"2023-11-24T06:15:51.810683+0000","last_fullsized":"2023-11-24T06:15:51.810683+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_clean_scrub_stamp":"2023-11-24T06:15:40.694368+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:41.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792437+0000","last_change":"2023-11-24T06:15:51.161936+0000","last_active":"2023-11-24T06:15:52.792437+0000","last_peered":"2023-11-24T06:15:52.792437+0000","last_clean":"2023-11-24T06:15:52.792437+0000","last_became_active":"2023-11-24T06:15:51.161566+0000","last_became_peered":"2023-11-24T06:15:51.161566+0000","last_unstale":"2023-11-24T06:15:52.792437+0000","last_undegraded":"2023-11-24T06:15:52.792437+0000","last_fullsized":"2023-11-24T06:15:52.792437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:42.342474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845108+0000","last_change":"2023-11-24T06:15:51.816547+0000","last_active":"2023-11-24T06:15:53.845108+0000","last_peered":"2023-11-24T06:15:53.845108+0000","last_clean":"2023-11-24T06:15:53.845108+0000","last_became_active":"2023-11-24T06:15:51.816231+0000","last_became_peered":"2023-11-24T06:15:51.816231+0000","last_unstale":"2023-11-24T06:15:53.845108+0000","last_undegraded":"2023-11-24T06:15:53.845108+0000","last_fullsized":"2023-11-24T06:15:53.845108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:21:49.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183719+0000","last_change":"2023-11-24T06:15:51.164362+0000","last_active":"2023-11-24T06:15:54.183719+0000","last_peered":"2023-11-24T06:15:54.183719+0000","last_clean":"2023-11-24T06:15:54.183719+0000","last_became_active":"2023-11-24T06:15:51.163606+0000","last_became_peered":"2023-11-24T06:15:51.163606+0000","last_unstale":"2023-11-24T06:15:54.183719+0000","last_undegraded":"2023-11-24T06:15:54.183719+0000","last_fullsized":"2023-11-24T06:15:54.183719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:36.901341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.824729+0000","last_change":"2023-11-24T06:15:51.821396+0000","last_active":"2023-11-24T06:15:51.824729+0000","last_peered":"2023-11-24T06:15:51.824729+0000","last_clean":"2023-11-24T06:15:51.824729+0000","last_became_active":"2023-11-24T06:15:51.821096+0000","last_became_peered":"2023-11-24T06:15:51.821096+0000","last_unstale":"2023-11-24T06:15:51.824729+0000","last_undegraded":"2023-11-24T06:15:51.824729+0000","last_fullsized":"2023-11-24T06:15:51.824729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:38.032032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.140207+0000","last_change":"2023-11-24T06:15:51.156378+0000","last_active":"2023-11-24T06:15:53.140207+0000","last_peered":"2023-11-24T06:15:53.140207+0000","last_clean":"2023-11-24T06:15:53.140207+0000","last_became_active":"2023-11-24T06:15:51.156003+0000","last_became_peered":"2023-11-24T06:15:51.156003+0000","last_unstale":"2023-11-24T06:15:53.140207+0000","last_undegraded":"2023-11-24T06:15:53.140207+0000","last_fullsized":"2023-11-24T06:15:53.140207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:31.267818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788625+0000","last_change":"2023-11-24T06:15:51.810053+0000","last_active":"2023-11-24T06:15:52.788625+0000","last_peered":"2023-11-24T06:15:52.788625+0000","last_clean":"2023-11-24T06:15:52.788625+0000","last_became_active":"2023-11-24T06:15:51.806554+0000","last_became_peered":"2023-11-24T06:15:51.806554+0000","last_unstale":"2023-11-24T06:15:52.788625+0000","last_undegraded":"2023-11-24T06:15:52.788625+0000","last_fullsized":"2023-11-24T06:15:52.788625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:01.976941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793809+0000","last_change":"2023-11-24T06:15:51.170982+0000","last_active":"2023-11-24T06:15:52.793809+0000","last_peered":"2023-11-24T06:15:52.793809+0000","last_clean":"2023-11-24T06:15:52.793809+0000","last_became_active":"2023-11-24T06:15:51.170654+0000","last_became_peered":"2023-11-24T06:15:51.170654+0000","last_unstale":"2023-11-24T06:15:52.793809+0000","last_undegraded":"2023-11-24T06:15:52.793809+0000","last_fullsized":"2023-11-24T06:15:52.793809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:48.317425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792676+0000","last_change":"2023-11-24T06:15:51.823915+0000","last_active":"2023-11-24T06:15:52.792676+0000","last_peered":"2023-11-24T06:15:52.792676+0000","last_clean":"2023-11-24T06:15:52.792676+0000","last_became_active":"2023-11-24T06:15:51.823654+0000","last_became_peered":"2023-11-24T06:15:51.823654+0000","last_unstale":"2023-11-24T06:15:52.792676+0000","last_undegraded":"2023-11-24T06:15:52.792676+0000","last_fullsized":"2023-11-24T06:15:52.792676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:17:01.717621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789034+0000","last_change":"2023-11-24T06:15:51.154171+0000","last_active":"2023-11-24T06:15:52.789034+0000","last_peered":"2023-11-24T06:15:52.789034+0000","last_clean":"2023-11-24T06:15:52.789034+0000","last_became_active":"2023-11-24T06:15:51.153885+0000","last_became_peered":"2023-11-24T06:15:51.153885+0000","last_unstale":"2023-11-24T06:15:52.789034+0000","last_undegraded":"2023-11-24T06:15:52.789034+0000","last_fullsized":"2023-11-24T06:15:52.789034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:36:33.590350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793463+0000","last_change":"2023-11-24T06:15:51.809270+0000","last_active":"2023-11-24T06:15:52.793463+0000","last_peered":"2023-11-24T06:15:52.793463+0000","last_clean":"2023-11-24T06:15:52.793463+0000","last_became_active":"2023-11-24T06:15:51.808994+0000","last_became_peered":"2023-11-24T06:15:51.808994+0000","last_unstale":"2023-11-24T06:15:52.793463+0000","last_undegraded":"2023-11-24T06:15:52.793463+0000","last_fullsized":"2023-11-24T06:15:52.793463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:54.543519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185536+0000","last_change":"2023-11-24T06:15:51.166677+0000","last_active":"2023-11-24T06:15:54.185536+0000","last_peered":"2023-11-24T06:15:54.185536+0000","last_clean":"2023-11-24T06:15:54.185536+0000","last_became_active":"2023-11-24T06:15:51.164374+0000","last_became_peered":"2023-11-24T06:15:51.164374+0000","last_unstale":"2023-11-24T06:15:54.185536+0000","last_undegraded":"2023-11-24T06:15:54.185536+0000","last_fullsized":"2023-11-24T06:15:54.185536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:00:50.542922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788766+0000","last_change":"2023-11-24T06:15:51.808320+0000","last_active":"2023-11-24T06:15:52.788766+0000","last_peered":"2023-11-24T06:15:52.788766+0000","last_clean":"2023-11-24T06:15:52.788766+0000","last_became_active":"2023-11-24T06:15:51.804832+0000","last_became_peered":"2023-11-24T06:15:51.804832+0000","last_unstale":"2023-11-24T06:15:52.788766+0000","last_undegraded":"2023-11-24T06:15:52.788766+0000","last_fullsized":"2023-11-24T06:15:52.788766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:21:49.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789431+0000","last_change":"2023-11-24T06:15:51.168020+0000","last_active":"2023-11-24T06:15:52.789431+0000","last_peered":"2023-11-24T06:15:52.789431+0000","last_clean":"2023-11-24T06:15:52.789431+0000","last_became_active":"2023-11-24T06:15:51.167664+0000","last_became_peered":"2023-11-24T06:15:51.167664+0000","last_unstale":"2023-11-24T06:15:52.789431+0000","last_undegraded":"2023-11-24T06:15:52.789431+0000","last_fullsized":"2023-11-24T06:15:52.789431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:17:00.705564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845799+0000","last_change":"2023-11-24T06:15:51.816931+0000","last_active":"2023-11-24T06:15:53.845799+0000","last_peered":"2023-11-24T06:15:53.845799+0000","last_clean":"2023-11-24T06:15:53.845799+0000","last_became_active":"2023-11-24T06:15:51.816611+0000","last_became_peered":"2023-11-24T06:15:51.816611+0000","last_unstale":"2023-11-24T06:15:53.845799+0000","last_undegraded":"2023-11-24T06:15:53.845799+0000","last_fullsized":"2023-11-24T06:15:53.845799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:27.816377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750733+0000","last_change":"2023-11-24T06:15:51.172657+0000","last_active":"2023-11-24T06:15:53.750733+0000","last_peered":"2023-11-24T06:15:53.750733+0000","last_clean":"2023-11-24T06:15:53.750733+0000","last_became_active":"2023-11-24T06:15:51.172152+0000","last_became_peered":"2023-11-24T06:15:51.172152+0000","last_unstale":"2023-11-24T06:15:53.750733+0000","last_undegraded":"2023-11-24T06:15:53.750733+0000","last_fullsized":"2023-11-24T06:15:53.750733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:23:43.714712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.823916+0000","last_change":"2023-11-24T06:15:51.815689+0000","last_active":"2023-11-24T06:15:51.823916+0000","last_peered":"2023-11-24T06:15:51.823916+0000","last_clean":"2023-11-24T06:15:51.823916+0000","last_became_active":"2023-11-24T06:15:51.814742+0000","last_became_peered":"2023-11-24T06:15:51.814742+0000","last_unstale":"2023-11-24T06:15:51.823916+0000","last_undegraded":"2023-11-24T06:15:51.823916+0000","last_fullsized":"2023-11-24T06:15:51.823916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:54:46.540704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183694+0000","last_change":"2023-11-24T06:15:51.164574+0000","last_active":"2023-11-24T06:15:54.183694+0000","last_peered":"2023-11-24T06:15:54.183694+0000","last_clean":"2023-11-24T06:15:54.183694+0000","last_became_active":"2023-11-24T06:15:51.163261+0000","last_became_peered":"2023-11-24T06:15:51.163261+0000","last_unstale":"2023-11-24T06:15:54.183694+0000","last_undegraded":"2023-11-24T06:15:54.183694+0000","last_fullsized":"2023-11-24T06:15:54.183694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:19:30.567064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791508+0000","last_change":"2023-11-24T06:15:51.808104+0000","last_active":"2023-11-24T06:15:52.791508+0000","last_peered":"2023-11-24T06:15:52.791508+0000","last_clean":"2023-11-24T06:15:52.791508+0000","last_became_active":"2023-11-24T06:15:51.807649+0000","last_became_peered":"2023-11-24T06:15:51.807649+0000","last_unstale":"2023-11-24T06:15:52.791508+0000","last_undegraded":"2023-11-24T06:15:52.791508+0000","last_fullsized":"2023-11-24T06:15:52.791508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:47.469041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788424+0000","last_change":"2023-11-24T06:15:51.160561+0000","last_active":"2023-11-24T06:15:52.788424+0000","last_peered":"2023-11-24T06:15:52.788424+0000","last_clean":"2023-11-24T06:15:52.788424+0000","last_became_active":"2023-11-24T06:15:51.160266+0000","last_became_peered":"2023-11-24T06:15:51.160266+0000","last_unstale":"2023-11-24T06:15:52.788424+0000","last_undegraded":"2023-11-24T06:15:52.788424+0000","last_fullsized":"2023-11-24T06:15:52.788424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:09:38.300549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845022+0000","last_change":"2023-11-24T06:15:51.806779+0000","last_active":"2023-11-24T06:15:53.845022+0000","last_peered":"2023-11-24T06:15:53.845022+0000","last_clean":"2023-11-24T06:15:53.845022+0000","last_became_active":"2023-11-24T06:15:51.806429+0000","last_became_peered":"2023-11-24T06:15:51.806429+0000","last_unstale":"2023-11-24T06:15:53.845022+0000","last_undegraded":"2023-11-24T06:15:53.845022+0000","last_fullsized":"2023-11-24T06:15:53.845022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:01.976941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788974+0000","last_change":"2023-11-24T06:15:51.157354+0000","last_active":"2023-11-24T06:15:52.788974+0000","last_peered":"2023-11-24T06:15:52.788974+0000","last_clean":"2023-11-24T06:15:52.788974+0000","last_became_active":"2023-11-24T06:15:51.156981+0000","last_became_peered":"2023-11-24T06:15:51.156981+0000","last_unstale":"2023-11-24T06:15:52.788974+0000","last_undegraded":"2023-11-24T06:15:52.788974+0000","last_fullsized":"2023-11-24T06:15:52.788974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:19:30.567064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845730+0000","last_change":"2023-11-24T06:15:51.823678+0000","last_active":"2023-11-24T06:15:53.845730+0000","last_peered":"2023-11-24T06:15:53.845730+0000","last_clean":"2023-11-24T06:15:53.845730+0000","last_became_active":"2023-11-24T06:15:51.823372+0000","last_became_peered":"2023-11-24T06:15:51.823372+0000","last_unstale":"2023-11-24T06:15:53.845730+0000","last_undegraded":"2023-11-24T06:15:53.845730+0000","last_fullsized":"2023-11-24T06:15:53.845730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:46.484813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184891+0000","last_change":"2023-11-24T06:15:51.167493+0000","last_active":"2023-11-24T06:15:54.184891+0000","last_peered":"2023-11-24T06:15:54.184891+0000","last_clean":"2023-11-24T06:15:54.184891+0000","last_became_active":"2023-11-24T06:15:51.167110+0000","last_became_peered":"2023-11-24T06:15:51.167110+0000","last_unstale":"2023-11-24T06:15:54.184891+0000","last_undegraded":"2023-11-24T06:15:54.184891+0000","last_fullsized":"2023-11-24T06:15:54.184891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:59.444337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749958+0000","last_change":"2023-11-24T06:15:51.816423+0000","last_active":"2023-11-24T06:15:53.749958+0000","last_peered":"2023-11-24T06:15:53.749958+0000","last_clean":"2023-11-24T06:15:53.749958+0000","last_became_active":"2023-11-24T06:15:51.816122+0000","last_became_peered":"2023-11-24T06:15:51.816122+0000","last_unstale":"2023-11-24T06:15:53.749958+0000","last_undegraded":"2023-11-24T06:15:53.749958+0000","last_fullsized":"2023-11-24T06:15:53.749958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:50.460127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793429+0000","last_change":"2023-11-24T06:15:51.803878+0000","last_active":"2023-11-24T06:15:52.793429+0000","last_peered":"2023-11-24T06:15:52.793429+0000","last_clean":"2023-11-24T06:15:52.793429+0000","last_became_active":"2023-11-24T06:15:51.803472+0000","last_became_peered":"2023-11-24T06:15:51.803472+0000","last_unstale":"2023-11-24T06:15:52.793429+0000","last_undegraded":"2023-11-24T06:15:52.793429+0000","last_fullsized":"2023-11-24T06:15:52.793429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:23.431671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184531+0000","last_change":"2023-11-24T06:15:51.177433+0000","last_active":"2023-11-24T06:15:54.184531+0000","last_peered":"2023-11-24T06:15:54.184531+0000","last_clean":"2023-11-24T06:15:54.184531+0000","last_became_active":"2023-11-24T06:15:51.176987+0000","last_became_peered":"2023-11-24T06:15:51.176987+0000","last_unstale":"2023-11-24T06:15:54.184531+0000","last_undegraded":"2023-11-24T06:15:54.184531+0000","last_fullsized":"2023-11-24T06:15:54.184531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:18.981340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793828+0000","last_change":"2023-11-24T06:15:51.815123+0000","last_active":"2023-11-24T06:15:52.793828+0000","last_peered":"2023-11-24T06:15:52.793828+0000","last_clean":"2023-11-24T06:15:52.793828+0000","last_became_active":"2023-11-24T06:15:51.814778+0000","last_became_peered":"2023-11-24T06:15:51.814778+0000","last_unstale":"2023-11-24T06:15:52.793828+0000","last_undegraded":"2023-11-24T06:15:52.793828+0000","last_fullsized":"2023-11-24T06:15:52.793828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:12.167390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185069+0000","last_change":"2023-11-24T06:15:51.169354+0000","last_active":"2023-11-24T06:15:54.185069+0000","last_peered":"2023-11-24T06:15:54.185069+0000","last_clean":"2023-11-24T06:15:54.185069+0000","last_became_active":"2023-11-24T06:15:51.168916+0000","last_became_peered":"2023-11-24T06:15:51.168916+0000","last_unstale":"2023-11-24T06:15:54.185069+0000","last_undegraded":"2023-11-24T06:15:54.185069+0000","last_fullsized":"2023-11-24T06:15:54.185069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:15.261716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.824464+0000","last_change":"2023-11-24T06:15:51.818739+0000","last_active":"2023-11-24T06:15:51.824464+0000","last_peered":"2023-11-24T06:15:51.824464+0000","last_clean":"2023-11-24T06:15:51.824464+0000","last_became_active":"2023-11-24T06:15:51.817316+0000","last_became_peered":"2023-11-24T06:15:51.817316+0000","last_unstale":"2023-11-24T06:15:51.824464+0000","last_undegraded":"2023-11-24T06:15:51.824464+0000","last_fullsized":"2023-11-24T06:15:51.824464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:00:56.411126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845707+0000","last_change":"2023-11-24T06:15:51.152517+0000","last_active":"2023-11-24T06:15:53.845707+0000","last_peered":"2023-11-24T06:15:53.845707+0000","last_clean":"2023-11-24T06:15:53.845707+0000","last_became_active":"2023-11-24T06:15:51.152218+0000","last_became_peered":"2023-11-24T06:15:51.152218+0000","last_unstale":"2023-11-24T06:15:53.845707+0000","last_undegraded":"2023-11-24T06:15:53.845707+0000","last_fullsized":"2023-11-24T06:15:53.845707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:59.444337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845977+0000","last_change":"2023-11-24T06:15:51.817856+0000","last_active":"2023-11-24T06:15:53.845977+0000","last_peered":"2023-11-24T06:15:53.845977+0000","last_clean":"2023-11-24T06:15:53.845977+0000","last_became_active":"2023-11-24T06:15:51.817558+0000","last_became_peered":"2023-11-24T06:15:51.817558+0000","last_unstale":"2023-11-24T06:15:53.845977+0000","last_undegraded":"2023-11-24T06:15:53.845977+0000","last_fullsized":"2023-11-24T06:15:53.845977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:47.291087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.158748+0000","last_change":"2023-11-24T06:15:51.157608+0000","last_active":"2023-11-24T06:15:52.158748+0000","last_peered":"2023-11-24T06:15:52.158748+0000","last_clean":"2023-11-24T06:15:52.158748+0000","last_became_active":"2023-11-24T06:15:51.157330+0000","last_became_peered":"2023-11-24T06:15:51.157330+0000","last_unstale":"2023-11-24T06:15:52.158748+0000","last_undegraded":"2023-11-24T06:15:52.158748+0000","last_fullsized":"2023-11-24T06:15:52.158748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:39:05.215342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.861}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]}]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":78220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.405}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.738}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2410000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6879999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":221184,"data_stored":73406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.611}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2030000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7549999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1230000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5179999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1819999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":78184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.649}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.425}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":221184,"data_stored":73406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7370000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3560000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9930000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.23}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":78094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":78254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1819999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.397}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7010000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0219999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.903}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":811008,"data_stored":662081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.119}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.302}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:15:57.114 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:15:57.114 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-24T06:15:57.114 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-24T06:15:57.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-24T06:15:57.115 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-24T06:15:57.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.378+0000 7fd6a5e7c640 1 Processor -- start 2023-11-24T06:15:57.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.378+0000 7fd6a5e7c640 1 -- start start 2023-11-24T06:15:57.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.379+0000 7fd6a5e7c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fd6a006d520 0x7fd6a006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.380+0000 7fd6a5e7c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a015c360 con 0x7fd6a00ad720 2023-11-24T06:15:57.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.380+0000 7fd6a5e7c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a015faf0 con 0x7fd6a00abff0 2023-11-24T06:15:57.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.380+0000 7fd6a5e7c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a00b41a0 con 0x7fd6a006d520 2023-11-24T06:15:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.380+0000 7fd69effd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd6a00ad720 0x7fd6a015ecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56460/0 (socket says 172.21.15.5:56460) 2023-11-24T06:15:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.380+0000 7fd69effd640 1 -- 172.21.15.5:0/645116229 learned_addr learned my addr 172.21.15.5:0/645116229 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.380+0000 7fd69f7fe640 1 --2- 172.21.15.5:0/645116229 >> v2:172.21.15.6:3300/0 conn(0x7fd6a006d520 0x7fd6a006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.380+0000 7fd69e7fc640 1 -- 172.21.15.5:0/645116229 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698331852 0 0) 0x7fd6a015c360 con 0x7fd6a00ad720 2023-11-24T06:15:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.380+0000 7fd69e7fc640 1 -- 172.21.15.5:0/645116229 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd68c003520 con 0x7fd6a00ad720 2023-11-24T06:15:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.381+0000 7fd69e7fc640 1 -- 172.21.15.5:0/645116229 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1397775796 0 0) 0x7fd68c003520 con 0x7fd6a00ad720 2023-11-24T06:15:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.381+0000 7fd69e7fc640 1 -- 172.21.15.5:0/645116229 >> v2:172.21.15.6:3300/0 conn(0x7fd6a006d520 msgr2=0x7fd6a006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.381+0000 7fd69e7fc640 1 --2- 172.21.15.5:0/645116229 >> v2:172.21.15.6:3300/0 conn(0x7fd6a006d520 0x7fd6a006d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.381+0000 7fd69e7fc640 1 -- 172.21.15.5:0/645116229 >> v1:172.21.15.6:6789/0 conn(0x7fd6a00abff0 legacy=0x7fd6a015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.381+0000 7fd69e7fc640 1 -- 172.21.15.5:0/645116229 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a006de40 con 0x7fd6a00ad720 2023-11-24T06:15:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.381+0000 7fd69e7fc640 1 -- 172.21.15.5:0/645116229 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fd6880019f0 con 0x7fd6a00ad720 2023-11-24T06:15:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.381+0000 7fd69e7fc640 1 -- 172.21.15.5:0/645116229 >> v1:172.21.15.5:6789/0 conn(0x7fd6a00ad720 legacy=0x7fd6a015ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.381+0000 7fd69e7fc640 1 --2- 172.21.15.5:0/645116229 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd68c003970 0x7fd68c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.381+0000 7fd69e7fc640 1 --2- 172.21.15.5:0/645116229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd68c0047c0 0x7fd68c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.381+0000 7fd69e7fc640 1 --2- 172.21.15.5:0/645116229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd68c005180 0x7fd68c00fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.381+0000 7fd69e7fc640 1 -- 172.21.15.5:0/645116229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c010400 con 0x7fd68c005180 2023-11-24T06:15:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.381+0000 7fd69e7fc640 1 -- 172.21.15.5:0/645116229 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c010570 con 0x7fd68c0047c0 2023-11-24T06:15:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.381+0000 7fd69e7fc640 1 -- 172.21.15.5:0/645116229 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68c010730 con 0x7fd68c003970 2023-11-24T06:15:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.381+0000 7fd69ffff640 1 --2- 172.21.15.5:0/645116229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd68c005180 0x7fd68c00fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.382+0000 7fd69effd640 1 --2- 172.21.15.5:0/645116229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd68c0047c0 0x7fd68c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.382+0000 7fd69f7fe640 1 --2- 172.21.15.5:0/645116229 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd68c003970 0x7fd68c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.382+0000 7fd69ffff640 1 -- 172.21.15.5:0/645116229 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd68c003970 msgr2=0x7fd68c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.382+0000 7fd69ffff640 1 --2- 172.21.15.5:0/645116229 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd68c003970 0x7fd68c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.382+0000 7fd69ffff640 1 -- 172.21.15.5:0/645116229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd68c0047c0 msgr2=0x7fd68c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.382+0000 7fd69ffff640 1 --2- 172.21.15.5:0/645116229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd68c0047c0 0x7fd68c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.382+0000 7fd69ffff640 1 -- 172.21.15.5:0/645116229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd68c003520 con 0x7fd68c005180 2023-11-24T06:15:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.383+0000 7fd69ffff640 1 -- 172.21.15.5:0/645116229 reap_dead start 2023-11-24T06:15:57.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.383+0000 7fd69ffff640 1 --2- 172.21.15.5:0/645116229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd68c005180 0x7fd68c00fd90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd698001720 tx=0x7fd68c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4148a4070af04896 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:57.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.383+0000 7fd69e7fc640 1 -- 172.21.15.5:0/645116229 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6980052d0 con 0x7fd68c005180 2023-11-24T06:15:57.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.383+0000 7fd69e7fc640 1 -- 172.21.15.5:0/645116229 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69802e050 con 0x7fd68c005180 2023-11-24T06:15:57.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.383+0000 7fd69e7fc640 1 -- 172.21.15.5:0/645116229 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6980035f0 con 0x7fd68c005180 2023-11-24T06:15:57.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.384+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/645116229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd68c005180 msgr2=0x7fd68c00fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:57.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.384+0000 7fd6a5e7c640 1 --2- 172.21.15.5:0/645116229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd68c005180 0x7fd68c00fd90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd698001720 tx=0x7fd68c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.384+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/645116229 shutdown_connections 2023-11-24T06:15:57.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.384+0000 7fd6a5e7c640 1 --2- 172.21.15.5:0/645116229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd68c005180 0x7fd68c00fd90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.384+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/645116229 >> 172.21.15.5:0/645116229 conn(0x7fd6a00b0d80 msgr2=0x7fd6a00b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:57.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.384+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/645116229 shutdown_connections 2023-11-24T06:15:57.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.384+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/645116229 wait complete. 2023-11-24T06:15:57.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.386+0000 7fd6a5e7c640 1 Processor -- start 2023-11-24T06:15:57.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.386+0000 7fd6a5e7c640 1 -- start start 2023-11-24T06:15:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.387+0000 7fd6a5e7c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd68c005180 0x7fd6a015aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.387+0000 7fd6a5e7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0152220 0x7fd6a0146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.387+0000 7fd6a5e7c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6a00b2d80 0x7fd6a00b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.387+0000 7fd6a5e7c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a00ce790 con 0x7fd6a0152220 2023-11-24T06:15:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.387+0000 7fd6a5e7c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a00b3720 con 0x7fd6a00b2d80 2023-11-24T06:15:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.387+0000 7fd6a5e7c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a00cdb80 con 0x7fd68c005180 2023-11-24T06:15:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.387+0000 7fd69effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0152220 0x7fd6a0146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.387+0000 7fd69effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0152220 0x7fd6a0146170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35636/0 (socket says 172.21.15.5:35636) 2023-11-24T06:15:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.387+0000 7fd69effd640 1 -- 172.21.15.5:0/3340580884 learned_addr learned my addr 172.21.15.5:0/3340580884 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.387+0000 7fd69f7fe640 1 --2- 172.21.15.5:0/3340580884 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd68c005180 0x7fd6a015aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.388+0000 7fd69ffff640 1 --2- 172.21.15.5:0/3340580884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6a00b2d80 0x7fd6a00b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.388+0000 7fd69effd640 1 -- 172.21.15.5:0/3340580884 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd68c005180 msgr2=0x7fd6a015aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.388+0000 7fd69effd640 1 --2- 172.21.15.5:0/3340580884 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd68c005180 0x7fd6a015aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.388+0000 7fd69effd640 1 -- 172.21.15.5:0/3340580884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6a00b2d80 msgr2=0x7fd6a00b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.388+0000 7fd69effd640 1 --2- 172.21.15.5:0/3340580884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6a00b2d80 0x7fd6a00b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.388+0000 7fd69effd640 1 -- 172.21.15.5:0/3340580884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a00cde60 con 0x7fd6a0152220 2023-11-24T06:15:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.388+0000 7fd69effd640 1 --2- 172.21.15.5:0/3340580884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0152220 0x7fd6a0146170 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd6880079e0 tx=0x7fd68800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:57.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.388+0000 7fd697fff640 1 -- 172.21.15.5:0/3340580884 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68800ee50 con 0x7fd6a0152220 2023-11-24T06:15:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.388+0000 7fd697fff640 1 -- 172.21.15.5:0/3340580884 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd688013420 con 0x7fd6a0152220 2023-11-24T06:15:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.388+0000 7fd697fff640 1 -- 172.21.15.5:0/3340580884 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688013790 con 0x7fd6a0152220 2023-11-24T06:15:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.388+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/3340580884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a00ce140 con 0x7fd6a0152220 2023-11-24T06:15:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.388+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/3340580884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a00cec20 con 0x7fd6a0152220 2023-11-24T06:15:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.390+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/3340580884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a0074770 con 0x7fd6a0152220 2023-11-24T06:15:57.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.390+0000 7fd697fff640 1 -- 172.21.15.5:0/3340580884 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd6880138f0 con 0x7fd6a0152220 2023-11-24T06:15:57.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.391+0000 7fd697fff640 1 --2- 172.21.15.5:0/3340580884 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd68006ae40 0x7fd68006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.391+0000 7fd697fff640 1 -- 172.21.15.5:0/3340580884 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd688089da0 con 0x7fd6a0152220 2023-11-24T06:15:57.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.392+0000 7fd69f7fe640 1 --2- 172.21.15.5:0/3340580884 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd68006ae40 0x7fd68006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.392+0000 7fd69f7fe640 1 --2- 172.21.15.5:0/3340580884 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd68006ae40 0x7fd68006d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd690009a10 tx=0x7fd690008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.396+0000 7fd697fff640 1 -- 172.21.15.5:0/3340580884 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd688054db0 con 0x7fd6a0152220 2023-11-24T06:15:57.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.638+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/3340580884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd6a0074770 con 0x7fd6a0152220 2023-11-24T06:15:57.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.647+0000 7fd697fff640 1 -- 172.21.15.5:0/3340580884 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171025 (secure 0 0 0) 0x7fd6880583d0 con 0x7fd6a0152220 2023-11-24T06:15:57.651 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:15:57.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.652+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/3340580884 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd68006ae40 msgr2=0x7fd68006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:57.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.652+0000 7fd6a5e7c640 1 --2- 172.21.15.5:0/3340580884 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd68006ae40 0x7fd68006d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd690009a10 tx=0x7fd690008040 comp rx=0 tx=0).stop 2023-11-24T06:15:57.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.652+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/3340580884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0152220 msgr2=0x7fd6a0146170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:57.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.652+0000 7fd6a5e7c640 1 --2- 172.21.15.5:0/3340580884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0152220 0x7fd6a0146170 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd6880079e0 tx=0x7fd68800e530 comp rx=0 tx=0).stop 2023-11-24T06:15:57.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.652+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/3340580884 shutdown_connections 2023-11-24T06:15:57.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.652+0000 7fd6a5e7c640 1 --2- 172.21.15.5:0/3340580884 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd68006ae40 0x7fd68006d300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.652+0000 7fd6a5e7c640 1 --2- 172.21.15.5:0/3340580884 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd6a00b2d80 0x7fd6a00b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.652+0000 7fd6a5e7c640 1 --2- 172.21.15.5:0/3340580884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a0152220 0x7fd6a0146170 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.652+0000 7fd6a5e7c640 1 --2- 172.21.15.5:0/3340580884 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd68c005180 0x7fd6a015aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.652+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/3340580884 >> 172.21.15.5:0/3340580884 conn(0x7fd6a00b0d80 msgr2=0x7fd6a00af200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:57.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.652+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/3340580884 shutdown_connections 2023-11-24T06:15:57.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.652+0000 7fd6a5e7c640 1 -- 172.21.15.5:0/3340580884 wait complete. 2023-11-24T06:15:57.674 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"active_gid":4108,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":4230007794},{"type":"v1","addr":"172.21.15.5:6833","nonce":4230007794}]},"active_addr":"172.21.15.5:6833/4230007794","active_change":"2023-11-24T06:15:32.065522+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3261515704}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2194842364}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3654896767}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2591563213}]}]} 2023-11-24T06:15:57.678 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-24T06:15:57.678 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-24T06:15:57.678 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:15:57.779 INFO:tasks.ceph.mds.b.smithi006.stderr:2023-11-24T06:15:57.776+0000 7f9a7f223640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:15:57.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.903+0000 7f7c1dd1d640 1 Processor -- start 2023-11-24T06:15:57.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.903+0000 7f7c1dd1d640 1 -- start start 2023-11-24T06:15:57.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.904+0000 7f7c1dd1d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c1815a100 0x7f7c180b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.904+0000 7f7c1dd1d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c1813da30 con 0x7f7c180abff0 2023-11-24T06:15:57.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.904+0000 7f7c1dd1d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c1806e340 con 0x7f7c180ae7e0 2023-11-24T06:15:57.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.904+0000 7f7c1dd1d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c180ac840 con 0x7f7c1815a100 2023-11-24T06:15:57.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.905+0000 7f7c16ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7c180abff0 0x7f7c1806d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56476/0 (socket says 172.21.15.5:56476) 2023-11-24T06:15:57.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.905+0000 7f7c16ffd640 1 -- 172.21.15.5:0/3078645701 learned_addr learned my addr 172.21.15.5:0/3078645701 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.905+0000 7f7c177fe640 1 --2- 172.21.15.5:0/3078645701 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c1815a100 0x7f7c180b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.905+0000 7f7c167fc640 1 -- 172.21.15.5:0/3078645701 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177733074 0 0) 0x7f7c1813da30 con 0x7f7c180abff0 2023-11-24T06:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.905+0000 7f7c167fc640 1 -- 172.21.15.5:0/3078645701 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c04003520 con 0x7f7c180abff0 2023-11-24T06:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c167fc640 1 -- 172.21.15.5:0/3078645701 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2899923832 0 0) 0x7f7c04003520 con 0x7f7c180abff0 2023-11-24T06:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c167fc640 1 -- 172.21.15.5:0/3078645701 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c1815a100 msgr2=0x7f7c180b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c167fc640 1 --2- 172.21.15.5:0/3078645701 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c1815a100 0x7f7c180b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c167fc640 1 -- 172.21.15.5:0/3078645701 >> v1:172.21.15.6:6789/0 conn(0x7f7c180ae7e0 legacy=0x7f7c1806dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c167fc640 1 -- 172.21.15.5:0/3078645701 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c180ac560 con 0x7f7c180abff0 2023-11-24T06:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c167fc640 1 -- 172.21.15.5:0/3078645701 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f7c00001670 con 0x7f7c180abff0 2023-11-24T06:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c167fc640 1 -- 172.21.15.5:0/3078645701 >> v1:172.21.15.5:6789/0 conn(0x7f7c180abff0 legacy=0x7f7c1806d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c167fc640 1 --2- 172.21.15.5:0/3078645701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c040039a0 0x7f7c04003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c167fc640 1 --2- 172.21.15.5:0/3078645701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c040047c0 0x7f7c04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c167fc640 1 --2- 172.21.15.5:0/3078645701 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c1815a100 0x7f7c04005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c167fc640 1 -- 172.21.15.5:0/3078645701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c180ac840 con 0x7f7c040039a0 2023-11-24T06:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c167fc640 1 -- 172.21.15.5:0/3078645701 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0400bb10 con 0x7f7c040047c0 2023-11-24T06:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c167fc640 1 -- 172.21.15.5:0/3078645701 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0400bcd0 con 0x7f7c1815a100 2023-11-24T06:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c177fe640 1 --2- 172.21.15.5:0/3078645701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c040039a0 0x7f7c04003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c16ffd640 1 --2- 172.21.15.5:0/3078645701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c040047c0 0x7f7c04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c177fe640 1 -- 172.21.15.5:0/3078645701 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c1815a100 msgr2=0x7f7c04005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c177fe640 1 --2- 172.21.15.5:0/3078645701 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c1815a100 0x7f7c04005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c177fe640 1 -- 172.21.15.5:0/3078645701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c040047c0 msgr2=0x7f7c04004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c177fe640 1 --2- 172.21.15.5:0/3078645701 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c040047c0 0x7f7c04004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.906+0000 7f7c177fe640 1 -- 172.21.15.5:0/3078645701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c04003520 con 0x7f7c040039a0 2023-11-24T06:15:57.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.908+0000 7f7c177fe640 1 --2- 172.21.15.5:0/3078645701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c040039a0 0x7f7c04003dc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009fc0 tx=0x7f7c040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0380cd6fec82b09 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:57.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.908+0000 7f7c167fc640 1 -- 172.21.15.5:0/3078645701 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0802cc60 con 0x7f7c040039a0 2023-11-24T06:15:57.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.908+0000 7f7c167fc640 1 -- 172.21.15.5:0/3078645701 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0800f040 con 0x7f7c040039a0 2023-11-24T06:15:57.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.908+0000 7f7c167fc640 1 -- 172.21.15.5:0/3078645701 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c08034440 con 0x7f7c040039a0 2023-11-24T06:15:57.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.908+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/3078645701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c040039a0 msgr2=0x7f7c04003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:57.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.908+0000 7f7c1dd1d640 1 --2- 172.21.15.5:0/3078645701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c040039a0 0x7f7c04003dc0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7c08009fc0 tx=0x7f7c040057f0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.908+0000 7f7c17fff640 1 -- 172.21.15.5:0/3078645701 reap_dead start 2023-11-24T06:15:57.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.909+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/3078645701 shutdown_connections 2023-11-24T06:15:57.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.909+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/3078645701 >> 172.21.15.5:0/3078645701 conn(0x7f7c180b0d80 msgr2=0x7f7c180b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:57.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.909+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/3078645701 shutdown_connections 2023-11-24T06:15:57.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.909+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/3078645701 wait complete. 2023-11-24T06:15:57.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.910+0000 7f7c1dd1d640 1 Processor -- start 2023-11-24T06:15:57.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.911+0000 7f7c1dd1d640 1 -- start start 2023-11-24T06:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c1dd1d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c040039a0 0x7f7c1814ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c1dd1d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c1814bd90 0x7f7c180ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c1dd1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c1814c210 0x7f7c180caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c1dd1d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1815b350 con 0x7f7c1814c210 2023-11-24T06:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c1dd1d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c180c8650 con 0x7f7c040039a0 2023-11-24T06:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c1dd1d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1815aab0 con 0x7f7c1814bd90 2023-11-24T06:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c17fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c1814c210 0x7f7c180caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c17fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c1814c210 0x7f7c180caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35652/0 (socket says 172.21.15.5:35652) 2023-11-24T06:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c17fff640 1 -- 172.21.15.5:0/1206119079 learned_addr learned my addr 172.21.15.5:0/1206119079 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c177fe640 1 --2- 172.21.15.5:0/1206119079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c040039a0 0x7f7c1814ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c16ffd640 1 --2- 172.21.15.5:0/1206119079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c1814bd90 0x7f7c180ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c17fff640 1 -- 172.21.15.5:0/1206119079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c1814bd90 msgr2=0x7f7c180ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c17fff640 1 --2- 172.21.15.5:0/1206119079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c1814bd90 0x7f7c180ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c17fff640 1 -- 172.21.15.5:0/1206119079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c040039a0 msgr2=0x7f7c1814ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c17fff640 1 --2- 172.21.15.5:0/1206119079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c040039a0 0x7f7c1814ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.912+0000 7f7c17fff640 1 -- 172.21.15.5:0/1206119079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1815ad90 con 0x7f7c1814c210 2023-11-24T06:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.913+0000 7f7c17fff640 1 --2- 172.21.15.5:0/1206119079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c1814c210 0x7f7c180caa00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7c180b3e50 tx=0x7f7c1000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.913+0000 7f7c0ffff640 1 -- 172.21.15.5:0/1206119079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c100072e0 con 0x7f7c1814c210 2023-11-24T06:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.913+0000 7f7c0ffff640 1 -- 172.21.15.5:0/1206119079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c10007440 con 0x7f7c1814c210 2023-11-24T06:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.913+0000 7f7c0ffff640 1 -- 172.21.15.5:0/1206119079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10004020 con 0x7f7c1814c210 2023-11-24T06:15:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.913+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/1206119079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1815b070 con 0x7f7c1814c210 2023-11-24T06:15:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.913+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/1206119079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1815a6f0 con 0x7f7c1814c210 2023-11-24T06:15:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.914+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/1206119079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c180c8ae0 con 0x7f7c1814c210 2023-11-24T06:15:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.915+0000 7f7c0ffff640 1 -- 172.21.15.5:0/1206119079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c100041a0 con 0x7f7c1814c210 2023-11-24T06:15:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.916+0000 7f7c0ffff640 1 --2- 172.21.15.5:0/1206119079 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7bf806ae70 0x7f7bf806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.916+0000 7f7c0ffff640 1 -- 172.21.15.5:0/1206119079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c100886d0 con 0x7f7c1814c210 2023-11-24T06:15:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.916+0000 7f7c177fe640 1 --2- 172.21.15.5:0/1206119079 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7bf806ae70 0x7f7bf806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.917+0000 7f7c177fe640 1 --2- 172.21.15.5:0/1206119079 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7bf806ae70 0x7f7bf806d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7c08037040 tx=0x7f7c0802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:57.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:57.921+0000 7f7c0ffff640 1 -- 172.21.15.5:0/1206119079 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c180c8ae0 con 0x7f7c1814c210 2023-11-24T06:15:58.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.117+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/1206119079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c1815a880 con 0x7f7c1814c210 2023-11-24T06:15:58.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.119+0000 7f7c0ffff640 1 -- 172.21.15.5:0/1206119079 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16361 (secure 0 0 0) 0x7f7c10057210 con 0x7f7c1814c210 2023-11-24T06:15:58.122 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:15:58.123 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:52.779059+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:15:58.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.122+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/1206119079 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7bf806ae70 msgr2=0x7f7bf806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.122+0000 7f7c1dd1d640 1 --2- 172.21.15.5:0/1206119079 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7bf806ae70 0x7f7bf806d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7c08037040 tx=0x7f7c0802e000 comp rx=0 tx=0).stop 2023-11-24T06:15:58.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.122+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/1206119079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c1814c210 msgr2=0x7f7c180caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.122+0000 7f7c1dd1d640 1 --2- 172.21.15.5:0/1206119079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c1814c210 0x7f7c180caa00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7c180b3e50 tx=0x7f7c1000c820 comp rx=0 tx=0).stop 2023-11-24T06:15:58.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.122+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/1206119079 shutdown_connections 2023-11-24T06:15:58.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.122+0000 7f7c1dd1d640 1 --2- 172.21.15.5:0/1206119079 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7bf806ae70 0x7f7bf806d330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.122+0000 7f7c1dd1d640 1 --2- 172.21.15.5:0/1206119079 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c1814bd90 0x7f7c180ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.122+0000 7f7c1dd1d640 1 --2- 172.21.15.5:0/1206119079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c1814c210 0x7f7c180caa00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.122+0000 7f7c1dd1d640 1 --2- 172.21.15.5:0/1206119079 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c040039a0 0x7f7c1814ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.122+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/1206119079 >> 172.21.15.5:0/1206119079 conn(0x7f7c180b0d80 msgr2=0x7f7c180af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:58.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.122+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/1206119079 shutdown_connections 2023-11-24T06:15:58.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.122+0000 7f7c1dd1d640 1 -- 172.21.15.5:0/1206119079 wait complete. 2023-11-24T06:15:58.145 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-24T06:15:58.145 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:15:58.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.344+0000 7fa81e5d4640 1 Processor -- start 2023-11-24T06:15:58.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.345+0000 7fa81e5d4640 1 -- start start 2023-11-24T06:15:58.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.346+0000 7fa81e5d4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa818157770 0x7fa81806cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.347+0000 7fa81e5d4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa81815df80 0x7fa818070720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.347+0000 7fa81e5d4640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa818161670 con 0x7fa81806d080 2023-11-24T06:15:58.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.347+0000 7fa81e5d4640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81815e410 con 0x7fa818157770 2023-11-24T06:15:58.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.347+0000 7fa81e5d4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81806d500 con 0x7fa81815df80 2023-11-24T06:15:58.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.347+0000 7fa81cdd1640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa81806d080 0x7fa81815d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56478/0 (socket says 172.21.15.5:56478) 2023-11-24T06:15:58.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.347+0000 7fa81cdd1640 1 -- 172.21.15.5:0/309611064 learned_addr learned my addr 172.21.15.5:0/309611064 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:58.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.347+0000 7fa81d5d2640 1 --2- 172.21.15.5:0/309611064 >> v2:172.21.15.5:3300/0 conn(0x7fa818157770 0x7fa81806cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.347+0000 7fa81ddd3640 1 --2- 172.21.15.5:0/309611064 >> v2:172.21.15.6:3300/0 conn(0x7fa81815df80 0x7fa818070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.347+0000 7fa80ffff640 1 -- 172.21.15.5:0/309611064 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392788022 0 0) 0x7fa818161670 con 0x7fa81806d080 2023-11-24T06:15:58.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.347+0000 7fa80ffff640 1 -- 172.21.15.5:0/309611064 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa804003520 con 0x7fa81806d080 2023-11-24T06:15:58.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.348+0000 7fa81d5d2640 1 -- 172.21.15.5:0/309611064 >> v2:172.21.15.6:3300/0 conn(0x7fa81815df80 msgr2=0x7fa818070720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.348+0000 7fa81d5d2640 1 --2- 172.21.15.5:0/309611064 >> v2:172.21.15.6:3300/0 conn(0x7fa81815df80 0x7fa818070720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.348+0000 7fa81d5d2640 1 -- 172.21.15.5:0/309611064 >> v1:172.21.15.5:6789/0 conn(0x7fa81806d080 legacy=0x7fa81815d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.348+0000 7fa81d5d2640 1 -- 172.21.15.5:0/309611064 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81815e610 con 0x7fa818157770 2023-11-24T06:15:58.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.349+0000 7fa81d5d2640 1 --2- 172.21.15.5:0/309611064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa818157770 0x7fa81806cb40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa81806ca90 tx=0x7fa81402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0e67d3994e5073 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.349+0000 7fa80ffff640 1 -- 172.21.15.5:0/309611064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814031830 con 0x7fa818157770 2023-11-24T06:15:58.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.349+0000 7fa80ffff640 1 -- 172.21.15.5:0/309611064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa814031990 con 0x7fa818157770 2023-11-24T06:15:58.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.349+0000 7fa80ffff640 1 -- 172.21.15.5:0/309611064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814031d20 con 0x7fa818157770 2023-11-24T06:15:58.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.350+0000 7fa81e5d4640 1 -- 172.21.15.5:0/309611064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa818157770 msgr2=0x7fa81806cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.350+0000 7fa81e5d4640 1 --2- 172.21.15.5:0/309611064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa818157770 0x7fa81806cb40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa81806ca90 tx=0x7fa81402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.350+0000 7fa81e5d4640 1 -- 172.21.15.5:0/309611064 shutdown_connections 2023-11-24T06:15:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.350+0000 7fa81e5d4640 1 --2- 172.21.15.5:0/309611064 >> v2:172.21.15.6:3300/0 conn(0x7fa81815df80 0x7fa818070720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.350+0000 7fa81e5d4640 1 --2- 172.21.15.5:0/309611064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa818157770 0x7fa81806cb40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.350+0000 7fa81e5d4640 1 -- 172.21.15.5:0/309611064 >> 172.21.15.5:0/309611064 conn(0x7fa81806a860 msgr2=0x7fa81806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.350+0000 7fa81e5d4640 1 -- 172.21.15.5:0/309611064 shutdown_connections 2023-11-24T06:15:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.350+0000 7fa81e5d4640 1 -- 172.21.15.5:0/309611064 wait complete. 2023-11-24T06:15:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.352+0000 7fa81e5d4640 1 Processor -- start 2023-11-24T06:15:58.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.352+0000 7fa81e5d4640 1 -- start start 2023-11-24T06:15:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81e5d4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa81806d080 0x7fa8180ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81e5d4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa818157770 0x7fa8181444f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81e5d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa81815df80 0x7fa8180caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81e5d4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180cc870 con 0x7fa81815df80 2023-11-24T06:15:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81e5d4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180cbc60 con 0x7fa81806d080 2023-11-24T06:15:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81e5d4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8180cbf40 con 0x7fa818157770 2023-11-24T06:15:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81ddd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa81815df80 0x7fa8180caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81ddd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa81815df80 0x7fa8180caef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35660/0 (socket says 172.21.15.5:35660) 2023-11-24T06:15:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81ddd3640 1 -- 172.21.15.5:0/2604497872 learned_addr learned my addr 172.21.15.5:0/2604497872 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81d5d2640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa81806d080 0x7fa8180ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81cdd1640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa818157770 0x7fa8181444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81ddd3640 1 -- 172.21.15.5:0/2604497872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa818157770 msgr2=0x7fa8181444f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81ddd3640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa818157770 0x7fa8181444f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81ddd3640 1 -- 172.21.15.5:0/2604497872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa81806d080 msgr2=0x7fa8180ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81ddd3640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa81806d080 0x7fa8180ccd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81ddd3640 1 -- 172.21.15.5:0/2604497872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa814009400 con 0x7fa81815df80 2023-11-24T06:15:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.353+0000 7fa81cdd1640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa818157770 0x7fa8181444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:15:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.354+0000 7fa81ddd3640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa81815df80 0x7fa8180caef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa810007e40 tx=0x7fa81000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.354+0000 7fa80dffb640 1 -- 172.21.15.5:0/2604497872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa810014620 con 0x7fa81815df80 2023-11-24T06:15:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.354+0000 7fa80dffb640 1 -- 172.21.15.5:0/2604497872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa81000ee90 con 0x7fa81815df80 2023-11-24T06:15:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.354+0000 7fa81e5d4640 1 -- 172.21.15.5:0/2604497872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8180cc220 con 0x7fa81815df80 2023-11-24T06:15:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.354+0000 7fa80dffb640 1 -- 172.21.15.5:0/2604497872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8100149e0 con 0x7fa81815df80 2023-11-24T06:15:58.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.354+0000 7fa81e5d4640 1 -- 172.21.15.5:0/2604497872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa818144a30 con 0x7fa81815df80 2023-11-24T06:15:58.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.355+0000 7fa81e5d4640 1 -- 172.21.15.5:0/2604497872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8180cca20 con 0x7fa81815df80 2023-11-24T06:15:58.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.356+0000 7fa80dffb640 1 -- 172.21.15.5:0/2604497872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa810014b40 con 0x7fa81815df80 2023-11-24T06:15:58.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.357+0000 7fa80dffb640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa7fc06ae40 0x7fa7fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.357+0000 7fa80dffb640 1 -- 172.21.15.5:0/2604497872 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa81008a060 con 0x7fa81815df80 2023-11-24T06:15:58.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.358+0000 7fa81d5d2640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa7fc06ae40 0x7fa7fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.358+0000 7fa81d5d2640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa7fc06ae40 0x7fa7fc06d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa814036000 tx=0x7fa814008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.362+0000 7fa80dffb640 1 -- 172.21.15.5:0/2604497872 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8100bce90 con 0x7fa81815df80 2023-11-24T06:15:58.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.558+0000 7fa81e5d4640 1 -- 172.21.15.5:0/2604497872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa818066e80 con 0x7fa81815df80 2023-11-24T06:15:58.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.560+0000 7fa80dffb640 1 -- 172.21.15.5:0/2604497872 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16361 (secure 0 0 0) 0x7fa8180cca20 con 0x7fa81815df80 2023-11-24T06:15:58.564 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:15:58.564 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:52.779059+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:15:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.563+0000 7fa81e5d4640 1 -- 172.21.15.5:0/2604497872 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa7fc06ae40 msgr2=0x7fa7fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.563+0000 7fa81e5d4640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa7fc06ae40 0x7fa7fc06d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa814036000 tx=0x7fa814008030 comp rx=0 tx=0).stop 2023-11-24T06:15:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.563+0000 7fa81e5d4640 1 -- 172.21.15.5:0/2604497872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa81815df80 msgr2=0x7fa8180caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.563+0000 7fa81e5d4640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa81815df80 0x7fa8180caef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa810007e40 tx=0x7fa81000e4b0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.563+0000 7fa81e5d4640 1 -- 172.21.15.5:0/2604497872 shutdown_connections 2023-11-24T06:15:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.564+0000 7fa81e5d4640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa7fc06ae40 0x7fa7fc06d300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.564+0000 7fa81e5d4640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa818157770 0x7fa8181444f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.564+0000 7fa81e5d4640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa81815df80 0x7fa8180caef0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.564+0000 7fa81e5d4640 1 --2- 172.21.15.5:0/2604497872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa81806d080 0x7fa8180ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.564+0000 7fa81e5d4640 1 -- 172.21.15.5:0/2604497872 >> 172.21.15.5:0/2604497872 conn(0x7fa81806a860 msgr2=0x7fa8180cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.564+0000 7fa81e5d4640 1 -- 172.21.15.5:0/2604497872 shutdown_connections 2023-11-24T06:15:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.564+0000 7fa81e5d4640 1 -- 172.21.15.5:0/2604497872 wait complete. 2023-11-24T06:15:58.586 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-24T06:15:58.587 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-24T06:15:58.587 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-24T06:15:58.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-24T06:15:58.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-24T06:15:58.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-24T06:15:58.589 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-24T06:15:58.589 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-24T06:15:58.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.904+0000 7f0548d0c640 1 Processor -- start 2023-11-24T06:15:58.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.904+0000 7f0548d0c640 1 -- start start 2023-11-24T06:15:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.906+0000 7f0548d0c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f05440b2080 0x7f05440b2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.906+0000 7f0548d0c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f05440b3cb0 0x7f05440b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.906+0000 7f0548d0c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05440b2ee0 con 0x7f05440ae400 2023-11-24T06:15:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.906+0000 7f0548d0c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05440ae8e0 con 0x7f05440b2080 2023-11-24T06:15:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.906+0000 7f0548d0c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05440adbc0 con 0x7f05440b3cb0 2023-11-24T06:15:58.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.907+0000 7f054259c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f05440b2080 0x7f05440b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.907+0000 7f054259c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f05440b2080 0x7f05440b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35666/0 (socket says 172.21.15.5:35666) 2023-11-24T06:15:58.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.907+0000 7f054259c640 1 -- 172.21.15.5:0/1398193149 learned_addr learned my addr 172.21.15.5:0/1398193149 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:58.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.908+0000 7f054259c640 1 -- 172.21.15.5:0/1398193149 >> v2:172.21.15.6:3300/0 conn(0x7f05440b3cb0 msgr2=0x7f05440b29a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:15:58.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.908+0000 7fe010c6b640 1 Processor -- start 2023-11-24T06:15:58.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.908+0000 7f054259c640 1 --2- 172.21.15.5:0/1398193149 >> v2:172.21.15.6:3300/0 conn(0x7f05440b3cb0 0x7f05440b29a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.908+0000 7f054259c640 1 -- 172.21.15.5:0/1398193149 >> v1:172.21.15.6:6789/0 conn(0x7f05440ae400 legacy=0x7f05440acc20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:15:58.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.908+0000 7f054259c640 1 -- 172.21.15.5:0/1398193149 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05440720e0 con 0x7f05440b2080 2023-11-24T06:15:58.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.908+0000 7fe010c6b640 1 -- start start 2023-11-24T06:15:58.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.909+0000 7f054259c640 1 --2- 172.21.15.5:0/1398193149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05440b2080 0x7f05440b2460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0544071810 tx=0x7f053402b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9821392cfa0ff08 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.910+0000 7fe010c6b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe00c071fb0 0x7fe00c0adc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.910+0000 7fe010c6b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe00c15abe0 0x7fe00c0ae160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.910+0000 7fe010c6b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe00c15a8a0 con 0x7fe00c159d00 2023-11-24T06:15:58.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.910+0000 7fe010c6b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00c153f70 con 0x7fe00c15abe0 2023-11-24T06:15:58.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.910+0000 7fe010c6b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c0ae9e0 con 0x7fe00c071fb0 2023-11-24T06:15:58.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.910+0000 7f054159a640 1 -- 172.21.15.5:0/1398193149 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534031800 con 0x7f05440b2080 2023-11-24T06:15:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.910+0000 7f054159a640 1 -- 172.21.15.5:0/1398193149 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0534031960 con 0x7f05440b2080 2023-11-24T06:15:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.910+0000 7f054159a640 1 -- 172.21.15.5:0/1398193149 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534031cf0 con 0x7f05440b2080 2023-11-24T06:15:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.911+0000 7fe009d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe00c15abe0 0x7fe00c0ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.911+0000 7fe009d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe00c15abe0 0x7fe00c0ae160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35678/0 (socket says 172.21.15.5:35678) 2023-11-24T06:15:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.911+0000 7fe009d9b640 1 -- 172.21.15.5:0/1981090656 learned_addr learned my addr 172.21.15.5:0/1981090656 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.911+0000 7f0548d0c640 1 -- 172.21.15.5:0/1398193149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05440b2080 msgr2=0x7f05440b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.911+0000 7f0548d0c640 1 --2- 172.21.15.5:0/1398193149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05440b2080 0x7f05440b2460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0544071810 tx=0x7f053402b8a0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.911+0000 7f0548d0c640 1 -- 172.21.15.5:0/1398193149 shutdown_connections 2023-11-24T06:15:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.911+0000 7f0548d0c640 1 --2- 172.21.15.5:0/1398193149 >> v2:172.21.15.6:3300/0 conn(0x7f05440b3cb0 0x7f05440b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.911+0000 7fe00a59c640 1 --2- 172.21.15.5:0/1981090656 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe00c071fb0 0x7fe00c0adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.911+0000 7f0548d0c640 1 --2- 172.21.15.5:0/1398193149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05440b2080 0x7f05440b2460 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.911+0000 7f0548d0c640 1 -- 172.21.15.5:0/1398193149 >> 172.21.15.5:0/1398193149 conn(0x7f05440709f0 msgr2=0x7f05440718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.911+0000 7f0548d0c640 1 -- 172.21.15.5:0/1398193149 shutdown_connections 2023-11-24T06:15:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.912+0000 7f0548d0c640 1 -- 172.21.15.5:0/1398193149 wait complete. 2023-11-24T06:15:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.912+0000 7fe009d9b640 1 -- 172.21.15.5:0/1981090656 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe00c071fb0 msgr2=0x7fe00c0adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.912+0000 7fe009d9b640 1 --2- 172.21.15.5:0/1981090656 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe00c071fb0 0x7fe00c0adc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.912+0000 7fe009d9b640 1 -- 172.21.15.5:0/1981090656 >> v1:172.21.15.6:6789/0 conn(0x7fe00c159d00 legacy=0x7fe00c15a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.912+0000 7fe009d9b640 1 -- 172.21.15.5:0/1981090656 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00c06c550 con 0x7fe00c15abe0 2023-11-24T06:15:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.913+0000 7fe009d9b640 1 --2- 172.21.15.5:0/1981090656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00c15abe0 0x7fe00c0ae160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009870 tx=0x7fdff402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6aa2992609fbf0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.914+0000 7fe00959a640 1 -- 172.21.15.5:0/1981090656 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4031640 con 0x7fe00c15abe0 2023-11-24T06:15:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.914+0000 7fe00959a640 1 -- 172.21.15.5:0/1981090656 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff40317a0 con 0x7fe00c15abe0 2023-11-24T06:15:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.914+0000 7fe00959a640 1 -- 172.21.15.5:0/1981090656 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4031640 con 0x7fe00c15abe0 2023-11-24T06:15:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.914+0000 7f0548d0c640 1 Processor -- start 2023-11-24T06:15:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.915+0000 7f0548d0c640 1 -- start start 2023-11-24T06:15:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.915+0000 7f2f88768640 1 Processor -- start 2023-11-24T06:15:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.915+0000 7f2f88768640 1 -- start start 2023-11-24T06:15:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.915+0000 7fe010c6b640 1 -- 172.21.15.5:0/1981090656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00c15abe0 msgr2=0x7fe00c0ae160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.915+0000 7fe010c6b640 1 --2- 172.21.15.5:0/1981090656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00c15abe0 0x7fe00c0ae160 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009870 tx=0x7fdff402b8f0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.916+0000 7fe010c6b640 1 -- 172.21.15.5:0/1981090656 shutdown_connections 2023-11-24T06:15:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.916+0000 7fe010c6b640 1 --2- 172.21.15.5:0/1981090656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00c15abe0 0x7fe00c0ae160 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.916+0000 7fe010c6b640 1 --2- 172.21.15.5:0/1981090656 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe00c071fb0 0x7fe00c0adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.916+0000 7fe010c6b640 1 -- 172.21.15.5:0/1981090656 >> 172.21.15.5:0/1981090656 conn(0x7fe00c0709f0 msgr2=0x7fe00c0718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:58.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.916+0000 7fe010c6b640 1 -- 172.21.15.5:0/1981090656 shutdown_connections 2023-11-24T06:15:58.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.916+0000 7fe010c6b640 1 -- 172.21.15.5:0/1981090656 wait complete. 2023-11-24T06:15:58.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.917+0000 7f0548d0c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05440ae400 0x7f05440ce950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.917+0000 7f0548d0c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05440b3cb0 0x7f05440cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.917+0000 7f2f88768640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2f8015d770 0x7f2f800d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.917+0000 7f2f88768640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f800d3980 0x7f2f800b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.917+0000 7f2f88768640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f800d0b40 con 0x7f2f800d2740 2023-11-24T06:15:58.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.917+0000 7f2f88768640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f8015db50 con 0x7f2f8015d770 2023-11-24T06:15:58.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.917+0000 7f2f88768640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800d0860 con 0x7f2f800d3980 2023-11-24T06:15:58.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.917+0000 7f0548d0c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05440b1940 0x7f05441f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.918+0000 7f0548d0c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05440cd660 con 0x7f05440b1940 2023-11-24T06:15:58.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.918+0000 7f0548d0c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05440cd8b0 con 0x7f05440b3cb0 2023-11-24T06:15:58.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.918+0000 7f0548d0c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05440cdb90 con 0x7f05440ae400 2023-11-24T06:15:58.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.918+0000 7f0541d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05440b3cb0 0x7f05440cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.918+0000 7f0541d9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05440b3cb0 0x7f05440cd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:49944/0 (socket says 172.21.15.5:49944) 2023-11-24T06:15:58.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.918+0000 7f0541d9b640 1 -- 172.21.15.5:0/2807622637 learned_addr learned my addr 172.21.15.5:0/2807622637 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:58.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.918+0000 7f0542d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05440b1940 0x7f05441f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.918+0000 7f2f86d05640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f800d3980 0x7f2f800b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.919+0000 7f2f86504640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2f8015d770 0x7f2f800d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.919+0000 7f2f86504640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2f8015d770 0x7f2f800d2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:49946/0 (socket says 172.21.15.5:49946) 2023-11-24T06:15:58.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.919+0000 7fe010c6b640 1 Processor -- start 2023-11-24T06:15:58.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.919+0000 7f2f86504640 1 -- 172.21.15.5:0/4104519428 learned_addr learned my addr 172.21.15.5:0/4104519428 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:58.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.919+0000 7fe010c6b640 1 -- start start 2023-11-24T06:15:58.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.919+0000 7f0541d9b640 1 -- 172.21.15.5:0/2807622637 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05440ae400 msgr2=0x7f05440ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.919+0000 7f0541d9b640 1 --2- 172.21.15.5:0/2807622637 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f05440ae400 0x7f05440ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.919+0000 7f0541d9b640 1 -- 172.21.15.5:0/2807622637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05440b1940 msgr2=0x7f05441f97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.919+0000 7f2f85d03640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2f800d2740 0x7f2f800d3270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56486/0 (socket says 172.21.15.5:56486) 2023-11-24T06:15:58.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.919+0000 7f0541d9b640 1 --2- 172.21.15.5:0/2807622637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05440b1940 0x7f05441f97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.919+0000 7f0541d9b640 1 -- 172.21.15.5:0/2807622637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340094d0 con 0x7f05440b3cb0 2023-11-24T06:15:58.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.920+0000 7f0542d9d640 1 --2- 172.21.15.5:0/2807622637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05440b1940 0x7f05441f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:15:58.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.920+0000 7f2f85502640 1 -- 172.21.15.5:0/4104519428 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881387674 0 0) 0x7f2f800d0b40 con 0x7f2f800d2740 2023-11-24T06:15:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.920+0000 7f2f85502640 1 -- 172.21.15.5:0/4104519428 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f6c003520 con 0x7f2f800d2740 2023-11-24T06:15:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.921+0000 7f0541d9b640 1 --2- 172.21.15.5:0/2807622637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05440b3cb0 0x7f05440cd120 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f053c00ba50 tx=0x7f053c00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.922+0000 7f0532ffd640 1 -- 172.21.15.5:0/2807622637 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c00cc10 con 0x7f05440b3cb0 2023-11-24T06:15:58.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.922+0000 7f0532ffd640 1 -- 172.21.15.5:0/2807622637 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c00cd70 con 0x7f05440b3cb0 2023-11-24T06:15:58.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.922+0000 7f0532ffd640 1 -- 172.21.15.5:0/2807622637 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c014680 con 0x7f05440b3cb0 2023-11-24T06:15:58.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.921+0000 7fe010c6b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe00c071fb0 0x7fe00c14e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.921+0000 7fe010c6b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe00c159d00 0x7fe00c14c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.922+0000 7fe010c6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00c15abe0 0x7fe00c14cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.922+0000 7fe00a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe00c071fb0 0x7fe00c14e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.922+0000 7fe00a59c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe00c071fb0 0x7fe00c14e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:49962/0 (socket says 172.21.15.5:49962) 2023-11-24T06:15:58.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.922+0000 7fe00a59c640 1 -- 172.21.15.5:0/194067898 learned_addr learned my addr 172.21.15.5:0/194067898 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:58.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.922+0000 7fe010c6b640 1 -- 172.21.15.5:0/194067898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c14eaa0 con 0x7fe00c15abe0 2023-11-24T06:15:58.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.922+0000 7fe009d9b640 1 --2- 172.21.15.5:0/194067898 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe00c159d00 0x7fe00c14c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.922+0000 7fe010c6b640 1 -- 172.21.15.5:0/194067898 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c14d2c0 con 0x7fe00c071fb0 2023-11-24T06:15:58.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.922+0000 7fe010c6b640 1 -- 172.21.15.5:0/194067898 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00c0b0610 con 0x7fe00c159d00 2023-11-24T06:15:58.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.921+0000 7f2f86504640 1 -- 172.21.15.5:0/4104519428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f800d3980 msgr2=0x7f2f800b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.921+0000 7f2f86504640 1 --2- 172.21.15.5:0/4104519428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f800d3980 0x7f2f800b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.921+0000 7f2f86504640 1 -- 172.21.15.5:0/4104519428 >> v1:172.21.15.5:6789/0 conn(0x7f2f800d2740 legacy=0x7f2f800d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.921+0000 7f2f86504640 1 -- 172.21.15.5:0/4104519428 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f800d1720 con 0x7f2f8015d770 2023-11-24T06:15:58.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.923+0000 7f5a17fb0640 1 Processor -- start 2023-11-24T06:15:58.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.923+0000 7f0548d0c640 1 -- 172.21.15.5:0/2807622637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05440cde70 con 0x7f05440b3cb0 2023-11-24T06:15:58.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.923+0000 7f0548d0c640 1 -- 172.21.15.5:0/2807622637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05440ce430 con 0x7f05440b3cb0 2023-11-24T06:15:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.923+0000 7fe00ad9d640 1 --2- 172.21.15.5:0/194067898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00c15abe0 0x7fe00c14cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.924+0000 7f5a17fb0640 1 -- start start 2023-11-24T06:15:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.924+0000 7fb0bdc00640 1 Processor -- start 2023-11-24T06:15:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.924+0000 7fe00a59c640 1 -- 172.21.15.5:0/194067898 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe00c159d00 msgr2=0x7fe00c14c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.924+0000 7fe00a59c640 1 --2- 172.21.15.5:0/194067898 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe00c159d00 0x7fe00c14c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.924+0000 7fe00a59c640 1 -- 172.21.15.5:0/194067898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00c15abe0 msgr2=0x7fe00c14cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.924+0000 7fe00a59c640 1 --2- 172.21.15.5:0/194067898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe00c15abe0 0x7fe00c14cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.924+0000 7fe00a59c640 1 -- 172.21.15.5:0/194067898 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff40094d0 con 0x7fe00c071fb0 2023-11-24T06:15:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.924+0000 7f2f86504640 1 --2- 172.21.15.5:0/4104519428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f8015d770 0x7f2f800d2200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009870 tx=0x7f2f7c02b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12baf4639519e420 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.924+0000 7fb0bdc00640 1 -- start start 2023-11-24T06:15:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.924+0000 7f0548d0c640 1 -- 172.21.15.5:0/2807622637 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f05440b3790 con 0x7f05440b3cb0 2023-11-24T06:15:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.924+0000 7f2f85502640 1 -- 172.21.15.5:0/4104519428 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c030880 con 0x7f2f8015d770 2023-11-24T06:15:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.925+0000 7f2f85502640 1 -- 172.21.15.5:0/4104519428 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c0309e0 con 0x7f2f8015d770 2023-11-24T06:15:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.925+0000 7f2f85502640 1 -- 172.21.15.5:0/4104519428 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c030db0 con 0x7f2f8015d770 2023-11-24T06:15:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.925+0000 7fe00a59c640 1 --2- 172.21.15.5:0/194067898 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe00c071fb0 0x7fe00c14e560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00c850 tx=0x7fdffc00cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.925+0000 7fe002ffd640 1 -- 172.21.15.5:0/194067898 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc004020 con 0x7fe00c071fb0 2023-11-24T06:15:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.925+0000 7fe010c6b640 1 -- 172.21.15.5:0/194067898 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00c14d4c0 con 0x7fe00c071fb0 2023-11-24T06:15:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.925+0000 7fe010c6b640 1 -- 172.21.15.5:0/194067898 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00c14d850 con 0x7fe00c071fb0 2023-11-24T06:15:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.925+0000 7f2f88768640 1 -- 172.21.15.5:0/4104519428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f8015d770 msgr2=0x7f2f800d2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.925+0000 7f2f88768640 1 --2- 172.21.15.5:0/4104519428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f8015d770 0x7f2f800d2200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009870 tx=0x7f2f7c02b9c0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7f2f88768640 1 -- 172.21.15.5:0/4104519428 shutdown_connections 2023-11-24T06:15:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7f2f88768640 1 --2- 172.21.15.5:0/4104519428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f800d3980 0x7f2f800b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7f2f88768640 1 --2- 172.21.15.5:0/4104519428 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f8015d770 0x7f2f800d2200 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7f2f88768640 1 -- 172.21.15.5:0/4104519428 >> 172.21.15.5:0/4104519428 conn(0x7f2f800721a0 msgr2=0x7f2f8006b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7f5a17fb0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a10153af0 0x7f5a10071e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7fb0bdc00640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb0b815d770 0x7fb0b80d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7f5a17fb0640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5a1006d520 0x7f5a1006d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.927+0000 7f2f88768640 1 -- 172.21.15.5:0/4104519428 shutdown_connections 2023-11-24T06:15:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.927+0000 7f2f88768640 1 -- 172.21.15.5:0/4104519428 wait complete. 2023-11-24T06:15:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7f5a17fb0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1006dec0 con 0x7f5a100704b0 2023-11-24T06:15:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7f5a17fb0640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a10159a10 con 0x7f5a1006d520 2023-11-24T06:15:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7f5a17fb0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1015a7f0 con 0x7f5a10153af0 2023-11-24T06:15:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.927+0000 7f5a1554b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5a1006d520 0x7f5a1006d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.927+0000 7f5a15d4c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a10153af0 0x7f5a10071e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.927+0000 7f5a1554b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5a1006d520 0x7f5a1006d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:49986/0 (socket says 172.21.15.5:49986) 2023-11-24T06:15:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.927+0000 7f5a1654d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5a100704b0 0x7f5a10070940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56510/0 (socket says 172.21.15.5:56510) 2023-11-24T06:15:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7fb0bdc00640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb0b80d2740 0x7fb0b80d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7fb0bdc00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b80d0b40 con 0x7fb0b80d37b0 2023-11-24T06:15:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7fb0bdc00640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b80d3c60 con 0x7fb0b80d2740 2023-11-24T06:15:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.927+0000 7fb0bdc00640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b815db50 con 0x7fb0b815d770 2023-11-24T06:15:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.927+0000 7fb0b6ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb0b80d2740 0x7fb0b80d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.927+0000 7f5a1654d640 1 -- 172.21.15.5:0/2287529978 learned_addr learned my addr 172.21.15.5:0/2287529978 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.927+0000 7fb0b6ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb0b80d2740 0x7fb0b80d3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35708/0 (socket says 172.21.15.5:35708) 2023-11-24T06:15:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.927+0000 7fb0b6ffd640 1 -- 172.21.15.5:0/3264832238 learned_addr learned my addr 172.21.15.5:0/3264832238 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.927+0000 7fb0b77fe640 1 --2- 172.21.15.5:0/3264832238 >> v2:172.21.15.6:3300/0 conn(0x7fb0b815d770 0x7fb0b80d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.928+0000 7f5a15d4c640 1 -- 172.21.15.5:0/2287529978 >> v2:172.21.15.6:3300/0 conn(0x7f5a1006d520 msgr2=0x7f5a1006d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.928+0000 7f5a15d4c640 1 --2- 172.21.15.5:0/2287529978 >> v2:172.21.15.6:3300/0 conn(0x7f5a1006d520 0x7f5a1006d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.928+0000 7f5a15d4c640 1 -- 172.21.15.5:0/2287529978 >> v1:172.21.15.5:6789/0 conn(0x7f5a100704b0 legacy=0x7f5a10070940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.928+0000 7f5a15d4c640 1 -- 172.21.15.5:0/2287529978 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a10153ed0 con 0x7f5a10153af0 2023-11-24T06:15:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7fe002ffd640 1 -- 172.21.15.5:0/194067898 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc0041a0 con 0x7fe00c071fb0 2023-11-24T06:15:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.926+0000 7fe002ffd640 1 -- 172.21.15.5:0/194067898 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc0077d0 con 0x7fe00c071fb0 2023-11-24T06:15:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.929+0000 7f2f88768640 1 Processor -- start 2023-11-24T06:15:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.929+0000 7f0532ffd640 1 -- 172.21.15.5:0/2807622637 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f053c01d3b0 con 0x7f05440b3cb0 2023-11-24T06:15:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.928+0000 7fb0b6ffd640 1 -- 172.21.15.5:0/3264832238 >> v2:172.21.15.6:3300/0 conn(0x7fb0b815d770 msgr2=0x7fb0b80d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.928+0000 7fb0b6ffd640 1 --2- 172.21.15.5:0/3264832238 >> v2:172.21.15.6:3300/0 conn(0x7fb0b815d770 0x7fb0b80d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.928+0000 7fb0b6ffd640 1 -- 172.21.15.5:0/3264832238 >> v1:172.21.15.5:6789/0 conn(0x7fb0b80d37b0 legacy=0x7fb0b80b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.928+0000 7fb0b6ffd640 1 -- 172.21.15.5:0/3264832238 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b80d0860 con 0x7fb0b80d2740 2023-11-24T06:15:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.930+0000 7f2f88768640 1 -- start start 2023-11-24T06:15:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.930+0000 7fe010c6b640 1 -- 172.21.15.5:0/194067898 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdff0000fc0 con 0x7fe00c071fb0 2023-11-24T06:15:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.930+0000 7fb0b6ffd640 1 --2- 172.21.15.5:0/3264832238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b80d2740 0x7fb0b80d3270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80d0e00 tx=0x7fb0a802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0bfc66cf9fc891f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.931+0000 7fb0b67fc640 1 -- 172.21.15.5:0/3264832238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8031870 con 0x7fb0b80d2740 2023-11-24T06:15:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.931+0000 7fb0b67fc640 1 -- 172.21.15.5:0/3264832238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a80319d0 con 0x7fb0b80d2740 2023-11-24T06:15:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.931+0000 7fb0b67fc640 1 -- 172.21.15.5:0/3264832238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8031870 con 0x7fb0b80d2740 2023-11-24T06:15:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.931+0000 7f2f88768640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f800d3980 0x7f2f800cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.931+0000 7f2f88768640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f8015d770 0x7f2f800ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.931+0000 7f2f88768640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f8015cc60 0x7f2f800ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.931+0000 7f5a15d4c640 1 --2- 172.21.15.5:0/2287529978 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a10153af0 0x7f5a10071e50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5a080060f0 tx=0x7f5a0802e210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ced273b37a0db106 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.931+0000 7f0532ffd640 1 --2- 172.21.15.5:0/2807622637 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f052406ae40 0x7f052406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.931+0000 7f0532ffd640 1 -- 172.21.15.5:0/2807622637 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f053c089d40 con 0x7f05440b3cb0 2023-11-24T06:15:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.931+0000 7f0532ffd640 1 --2- 172.21.15.5:0/2807622637 >> [v2:172.21.15.5:6816/1571096651,v1:172.21.15.5:6817/1571096651] conn(0x7f0524073060 0x7f0524075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.931+0000 7f0532ffd640 1 -- 172.21.15.5:0/2807622637 --> [v2:172.21.15.5:6816/1571096651,v1:172.21.15.5:6817/1571096651] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0524075b30 con 0x7f0524073060 2023-11-24T06:15:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.931+0000 7f0532ffd640 1 -- 172.21.15.5:0/2807622637 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f053c08a0f0 con 0x7f05440b3cb0 2023-11-24T06:15:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7f0542d9d640 1 --2- 172.21.15.5:0/2807622637 >> [v2:172.21.15.5:6816/1571096651,v1:172.21.15.5:6817/1571096651] conn(0x7f0524073060 0x7f0524075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7f2f88768640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800c5a50 con 0x7f2f800d3980 2023-11-24T06:15:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7f2f88768640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800c6260 con 0x7f2f8015d770 2023-11-24T06:15:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7f2f88768640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f800c5ca0 con 0x7f2f8015cc60 2023-11-24T06:15:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7f5a14d4a640 1 -- 172.21.15.5:0/2287529978 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08036440 con 0x7f5a10153af0 2023-11-24T06:15:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7f5a14d4a640 1 -- 172.21.15.5:0/2287529978 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a080365a0 con 0x7f5a10153af0 2023-11-24T06:15:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7fb0bdc00640 1 -- 172.21.15.5:0/3264832238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b80d2740 msgr2=0x7fb0b80d3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7fb0bdc00640 1 --2- 172.21.15.5:0/3264832238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b80d2740 0x7fb0b80d3270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80d0e00 tx=0x7fb0a802b910 comp rx=0 tx=0).stop 2023-11-24T06:15:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7f5a14d4a640 1 -- 172.21.15.5:0/2287529978 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08036440 con 0x7f5a10153af0 2023-11-24T06:15:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7f2f86d05640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f8015cc60 0x7f2f800ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7fc451aa6640 1 Processor -- start 2023-11-24T06:15:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7f2f86504640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f800d3980 0x7f2f800cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7f2f86d05640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f8015cc60 0x7f2f800ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:53320/0 (socket says 172.21.15.5:53320) 2023-11-24T06:15:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7f2f86d05640 1 -- 172.21.15.5:0/1155464204 learned_addr learned my addr 172.21.15.5:0/1155464204 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7f2f85d03640 1 --2- 172.21.15.5:0/1155464204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f8015d770 0x7f2f800ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7fc451aa6640 1 -- start start 2023-11-24T06:15:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.933+0000 7fb0bdc00640 1 -- 172.21.15.5:0/3264832238 shutdown_connections 2023-11-24T06:15:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.933+0000 7fb0bdc00640 1 --2- 172.21.15.5:0/3264832238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b80d2740 0x7fb0b80d3270 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.933+0000 7fb0bdc00640 1 --2- 172.21.15.5:0/3264832238 >> v2:172.21.15.6:3300/0 conn(0x7fb0b815d770 0x7fb0b80d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.933+0000 7fb0bdc00640 1 -- 172.21.15.5:0/3264832238 >> 172.21.15.5:0/3264832238 conn(0x7fb0b80721a0 msgr2=0x7fb0b806b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:58.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.933+0000 7f5a17fb0640 1 -- 172.21.15.5:0/2287529978 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a10153af0 msgr2=0x7f5a10071e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.933+0000 7f5a17fb0640 1 --2- 172.21.15.5:0/2287529978 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a10153af0 0x7f5a10071e50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5a080060f0 tx=0x7f5a0802e210 comp rx=0 tx=0).stop 2023-11-24T06:15:58.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.933+0000 7f2f86504640 1 -- 172.21.15.5:0/1155464204 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f8015cc60 msgr2=0x7f2f800ca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.933+0000 7f2f86504640 1 --2- 172.21.15.5:0/1155464204 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f8015cc60 0x7f2f800ca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.933+0000 7f2f86504640 1 -- 172.21.15.5:0/1155464204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f8015d770 msgr2=0x7f2f800ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.933+0000 7f2f86504640 1 --2- 172.21.15.5:0/1155464204 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f8015d770 0x7f2f800ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.933+0000 7f2f86504640 1 -- 172.21.15.5:0/1155464204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c0094d0 con 0x7f2f800d3980 2023-11-24T06:15:58.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.932+0000 7f054259c640 1 --2- 172.21.15.5:0/2807622637 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f052406ae40 0x7f052406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.934+0000 7f054259c640 1 --2- 172.21.15.5:0/2807622637 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f052406ae40 0x7f052406d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0534036000 tx=0x7f0534030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.934+0000 7fb0bdc00640 1 -- 172.21.15.5:0/3264832238 shutdown_connections 2023-11-24T06:15:58.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.934+0000 7f2f86504640 1 --2- 172.21.15.5:0/1155464204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f800d3980 0x7f2f800cbe40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c033000 tx=0x7f2f7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.934+0000 7f0542d9d640 1 --2- 172.21.15.5:0/2807622637 >> [v2:172.21.15.5:6816/1571096651,v1:172.21.15.5:6817/1571096651] conn(0x7f0524073060 0x7f0524075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.934+0000 7f5a17fb0640 1 -- 172.21.15.5:0/2287529978 shutdown_connections 2023-11-24T06:15:58.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.934+0000 7f5a17fb0640 1 --2- 172.21.15.5:0/2287529978 >> v2:172.21.15.6:3300/0 conn(0x7f5a1006d520 0x7f5a1006d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.934+0000 7f5a17fb0640 1 --2- 172.21.15.5:0/2287529978 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a10153af0 0x7f5a10071e50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.934+0000 7f5a17fb0640 1 -- 172.21.15.5:0/2287529978 >> 172.21.15.5:0/2287529978 conn(0x7f5a100b0d80 msgr2=0x7f5a100b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:58.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.934+0000 7fc451aa6640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc44c15d770 0x7fc44c0d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.934+0000 7fc451aa6640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc44c0d2740 0x7fc44c0d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.935+0000 7fc451aa6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc44c0d37b0 0x7fc44c0b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.935+0000 7fc451aa6640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c0d3c30 con 0x7fc44c0d37b0 2023-11-24T06:15:58.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.935+0000 7fc451aa6640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c0d0b40 con 0x7fc44c0d2740 2023-11-24T06:15:58.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.935+0000 7fc451aa6640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c15db50 con 0x7fc44c15d770 2023-11-24T06:15:58.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.934+0000 7f5a17fb0640 1 -- 172.21.15.5:0/2287529978 shutdown_connections 2023-11-24T06:15:58.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.934+0000 7f5a17fb0640 1 -- 172.21.15.5:0/2287529978 wait complete. 2023-11-24T06:15:58.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.935+0000 7fc44bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc44c0d37b0 0x7fc44c0b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.935+0000 7fc44bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc44c0d37b0 0x7fc44c0b6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35718/0 (socket says 172.21.15.5:35718) 2023-11-24T06:15:58.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.935+0000 7fc44bfff640 1 -- 172.21.15.5:0/2902186267 learned_addr learned my addr 172.21.15.5:0/2902186267 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:58.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.935+0000 7fc443fff640 1 --2- 172.21.15.5:0/2902186267 >> v2:172.21.15.6:3300/0 conn(0x7fc44c0d2740 0x7fc44c0d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7f0532ffd640 1 -- 172.21.15.5:0/2807622637 <== osd.0 v2:172.21.15.5:6816/1571096651 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0524075b30 con 0x7f0524073060 2023-11-24T06:15:58.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7fc44bfff640 1 -- 172.21.15.5:0/2902186267 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc44c15d770 msgr2=0x7fc44c0d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7fc44bfff640 1 --2- 172.21.15.5:0/2902186267 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc44c15d770 0x7fc44c0d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7fc44bfff640 1 -- 172.21.15.5:0/2902186267 >> v2:172.21.15.6:3300/0 conn(0x7fc44c0d2740 msgr2=0x7fc44c0d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7fc44bfff640 1 --2- 172.21.15.5:0/2902186267 >> v2:172.21.15.6:3300/0 conn(0x7fc44c0d2740 0x7fc44c0d3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7fc44bfff640 1 -- 172.21.15.5:0/2902186267 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c0d0860 con 0x7fc44c0d37b0 2023-11-24T06:15:58.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7fd965bb0640 1 Processor -- start 2023-11-24T06:15:58.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7fc443fff640 1 --2- 172.21.15.5:0/2902186267 >> v2:172.21.15.6:3300/0 conn(0x7fc44c0d2740 0x7fc44c0d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:15:58.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7fb0bdc00640 1 -- 172.21.15.5:0/3264832238 wait complete. 2023-11-24T06:15:58.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7fd965bb0640 1 -- start start 2023-11-24T06:15:58.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.934+0000 7fe002ffd640 1 -- 172.21.15.5:0/194067898 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdffc007930 con 0x7fe00c071fb0 2023-11-24T06:15:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7fe002ffd640 1 --2- 172.21.15.5:0/194067898 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdfec06ae70 0x7fdfec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7fe002ffd640 1 -- 172.21.15.5:0/194067898 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdffc0898b0 con 0x7fe00c071fb0 2023-11-24T06:15:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7fe002ffd640 1 --2- 172.21.15.5:0/194067898 >> [v2:172.21.15.5:6800/1318127149,v1:172.21.15.5:6801/1318127149] conn(0x7fdfec073090 0x7fdfec0754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7fe002ffd640 1 -- 172.21.15.5:0/194067898 --> [v2:172.21.15.5:6800/1318127149,v1:172.21.15.5:6801/1318127149] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfec075b60 con 0x7fdfec073090 2023-11-24T06:15:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.936+0000 7fe002ffd640 1 -- 172.21.15.5:0/194067898 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdffc089c60 con 0x7fe00c071fb0 2023-11-24T06:15:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.937+0000 7fe00ad9d640 1 --2- 172.21.15.5:0/194067898 >> [v2:172.21.15.5:6800/1318127149,v1:172.21.15.5:6801/1318127149] conn(0x7fdfec073090 0x7fdfec0754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.937+0000 7f0948e56640 1 Processor -- start 2023-11-24T06:15:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.938+0000 7f0948e56640 1 -- start start 2023-11-24T06:15:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.938+0000 7fe00ad9d640 1 --2- 172.21.15.5:0/194067898 >> [v2:172.21.15.5:6800/1318127149,v1:172.21.15.5:6801/1318127149] conn(0x7fdfec073090 0x7fdfec0754b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.938+0000 7f5a17fb0640 1 Processor -- start 2023-11-24T06:15:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.938+0000 7fc44bfff640 1 --2- 172.21.15.5:0/2902186267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c0d37b0 0x7fc44c0b6760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0d43e0 tx=0x7fc44402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd98055198ca080 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.938+0000 7fd965bb0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd96015d770 0x7fd9600d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.938+0000 7f2f76ffd640 1 -- 172.21.15.5:0/1155464204 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c039a10 con 0x7f2f800d3980 2023-11-24T06:15:58.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.938+0000 7f2f76ffd640 1 -- 172.21.15.5:0/1155464204 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c039b70 con 0x7f2f800d3980 2023-11-24T06:15:58.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.938+0000 7f2f76ffd640 1 -- 172.21.15.5:0/1155464204 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c0433f0 con 0x7f2f800d3980 2023-11-24T06:15:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.938+0000 7fd965bb0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9600d0b40 con 0x7fd9600d3980 2023-11-24T06:15:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.938+0000 7fd965bb0640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd96015db50 con 0x7fd9600d2740 2023-11-24T06:15:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.938+0000 7fd965bb0640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9600d0860 con 0x7fd96015d770 2023-11-24T06:15:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.938+0000 7f2f88768640 1 -- 172.21.15.5:0/1155464204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f800c5f80 con 0x7f2f800d3980 2023-11-24T06:15:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.938+0000 7fe009d9b640 1 --2- 172.21.15.5:0/194067898 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdfec06ae70 0x7fdfec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.938+0000 7fb0bdc00640 1 Processor -- start 2023-11-24T06:15:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.939+0000 7f2f88768640 1 -- 172.21.15.5:0/1155464204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f800cc380 con 0x7f2f800d3980 2023-11-24T06:15:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.939+0000 7fd95f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd96015d770 0x7fd9600d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.939+0000 7fd95effd640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fd9600d2740 0x7fd9600d3270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.5:45698/0 (socket says 172.21.15.5:45698) 2023-11-24T06:15:58.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.939+0000 7fd95effd640 1 -- 172.21.15.5:0/2600666789 learned_addr learned my addr 172.21.15.5:0/2600666789 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:58.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.939+0000 7f0948e56640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f094415d770 0x7f09440d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.939+0000 7f0948e56640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09440d0b40 con 0x7f09440d3980 2023-11-24T06:15:58.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.939+0000 7f0948e56640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f094415db50 con 0x7f09440d2740 2023-11-24T06:15:58.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.939+0000 7f0948e56640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09440d0860 con 0x7f094415d770 2023-11-24T06:15:58.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.939+0000 7fb0bdc00640 1 -- start start 2023-11-24T06:15:58.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.940+0000 7fd95e7fc640 1 -- 172.21.15.5:0/2600666789 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836666274 0 0) 0x7fd96015db50 con 0x7fd9600d2740 2023-11-24T06:15:58.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.940+0000 7fd95e7fc640 1 -- 172.21.15.5:0/2600666789 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd948003520 con 0x7fd9600d2740 2023-11-24T06:15:58.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.940+0000 7f0941d9b640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f09440d2740 0x7f09440d3270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.5:45710/0 (socket says 172.21.15.5:45710) 2023-11-24T06:15:58.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.940+0000 7f0941d9b640 1 -- 172.21.15.5:0/563432871 learned_addr learned my addr 172.21.15.5:0/563432871 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.940+0000 7fc44affd640 1 -- 172.21.15.5:0/2902186267 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44402ca80 con 0x7fc44c0d37b0 2023-11-24T06:15:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.940+0000 7fc44affd640 1 -- 172.21.15.5:0/2902186267 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44402cbe0 con 0x7fc44c0d37b0 2023-11-24T06:15:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.940+0000 7fc44affd640 1 -- 172.21.15.5:0/2902186267 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4440343f0 con 0x7fc44c0d37b0 2023-11-24T06:15:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.940+0000 7fe002ffd640 1 -- 172.21.15.5:0/194067898 <== osd.1 v2:172.21.15.5:6800/1318127149 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fdfec075b60 con 0x7fdfec073090 2023-11-24T06:15:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7f2f88768640 1 -- 172.21.15.5:0/1155464204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2f800b43d0 con 0x7f2f800d3980 2023-11-24T06:15:58.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95e7fc640 1 -- 172.21.15.5:0/2600666789 <== mon.1 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4198787706 0 0) 0x7fd948003520 con 0x7fd9600d2740 2023-11-24T06:15:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95e7fc640 1 -- 172.21.15.5:0/2600666789 >> v2:172.21.15.5:3300/0 conn(0x7fd96015d770 msgr2=0x7fd9600d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95e7fc640 1 --2- 172.21.15.5:0/2600666789 >> v2:172.21.15.5:3300/0 conn(0x7fd96015d770 0x7fd9600d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95e7fc640 1 -- 172.21.15.5:0/2600666789 >> v1:172.21.15.5:6789/0 conn(0x7fd9600d3980 legacy=0x7fd9600b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95e7fc640 1 -- 172.21.15.5:0/2600666789 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9600d1720 con 0x7fd9600d2740 2023-11-24T06:15:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95e7fc640 1 -- 172.21.15.5:0/2600666789 <== mon.1 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fd954001670 con 0x7fd9600d2740 2023-11-24T06:15:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95e7fc640 1 -- 172.21.15.5:0/2600666789 >> v1:172.21.15.6:6789/0 conn(0x7fd9600d2740 legacy=0x7fd9600d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95e7fc640 1 --2- 172.21.15.5:0/2600666789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd948003970 0x7fd948003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95e7fc640 1 --2- 172.21.15.5:0/2600666789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9480047c0 0x7fd948004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95e7fc640 1 --2- 172.21.15.5:0/2600666789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd948005180 0x7fd94800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95e7fc640 1 -- 172.21.15.5:0/2600666789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600d0860 con 0x7fd9480047c0 2023-11-24T06:15:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95e7fc640 1 -- 172.21.15.5:0/2600666789 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94800c000 con 0x7fd948003970 2023-11-24T06:15:58.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95e7fc640 1 -- 172.21.15.5:0/2600666789 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94800c1c0 con 0x7fd948005180 2023-11-24T06:15:58.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95effd640 1 --2- 172.21.15.5:0/2600666789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9480047c0 0x7fd948004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.941+0000 7fd95f7fe640 1 --2- 172.21.15.5:0/2600666789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd948003970 0x7fd948003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.939+0000 7f5a17fb0640 1 -- start start 2023-11-24T06:15:58.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.942+0000 7f094259c640 1 --2- 172.21.15.5:0/563432871 >> v2:172.21.15.6:3300/0 conn(0x7f094415d770 0x7f09440d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.942+0000 7fd95effd640 1 -- 172.21.15.5:0/2600666789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd948005180 msgr2=0x7fd94800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.942+0000 7fd95effd640 1 --2- 172.21.15.5:0/2600666789 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd948005180 0x7fd94800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.942+0000 7fd95effd640 1 -- 172.21.15.5:0/2600666789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd948003970 msgr2=0x7fd948003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.942+0000 7fd95effd640 1 --2- 172.21.15.5:0/2600666789 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd948003970 0x7fd948003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.942+0000 7fd95effd640 1 -- 172.21.15.5:0/2600666789 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd948003520 con 0x7fd9480047c0 2023-11-24T06:15:58.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.942+0000 7fd95ffff640 1 -- 172.21.15.5:0/2600666789 reap_dead start 2023-11-24T06:15:58.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.942+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959866182 0 0) 0x7f094415db50 con 0x7f09440d2740 2023-11-24T06:15:58.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f092c003520 con 0x7f09440d2740 2023-11-24T06:15:58.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.942+0000 7fc451aa6640 1 -- 172.21.15.5:0/2902186267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c0d37b0 msgr2=0x7fc44c0b6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.942+0000 7fc451aa6640 1 --2- 172.21.15.5:0/2902186267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c0d37b0 0x7fc44c0b6760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc44c0d43e0 tx=0x7fc44402bee0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7fb0bdc00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0b80d37b0 0x7fb0b81f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.942+0000 7fe009d9b640 1 --2- 172.21.15.5:0/194067898 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdfec06ae70 0x7fdfec06d330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdff4036000 tx=0x7fdff403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454401653 0 0) 0x7f09440d0b40 con 0x7f09440d3980 2023-11-24T06:15:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f5a17fb0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a1006d520 0x7f5a1014be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f5a17fb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a100704b0 0x7f5a10098500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f094415db50 con 0x7f09440d3980 2023-11-24T06:15:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 <== mon.1 v1:172.21.15.6:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4038918872 0 0) 0x7f092c003520 con 0x7f09440d2740 2023-11-24T06:15:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 >> v2:172.21.15.6:3300/0 conn(0x7f094415d770 msgr2=0x7f09440d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f094159a640 1 --2- 172.21.15.5:0/563432871 >> v2:172.21.15.6:3300/0 conn(0x7f094415d770 0x7f09440d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 >> v1:172.21.15.5:6789/0 conn(0x7f09440d3980 legacy=0x7f09440b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 --> v1:172.21.15.6:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09440d1720 con 0x7f09440d2740 2023-11-24T06:15:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 <== mon.1 v1:172.21.15.6:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f0938001670 con 0x7f09440d2740 2023-11-24T06:15:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7fd95effd640 1 --2- 172.21.15.5:0/2600666789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9480047c0 0x7fd948004c40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd9600ca360 tx=0x7fd9480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b283fc98a46bd1e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 >> v1:172.21.15.6:6789/0 conn(0x7f09440d2740 legacy=0x7f09440d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f094159a640 1 --2- 172.21.15.5:0/563432871 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f092c003b00 0x7f092c003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f094159a640 1 --2- 172.21.15.5:0/563432871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f092c0047c0 0x7f092c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f5a17fb0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a10153af0 0x7f5a100ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f5a17fb0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a100ce870 con 0x7f5a100704b0 2023-11-24T06:15:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f5a17fb0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1006e6d0 con 0x7f5a10153af0 2023-11-24T06:15:58.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7f5a17fb0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a10150fc0 con 0x7f5a1006d520 2023-11-24T06:15:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7f094159a640 1 --2- 172.21.15.5:0/563432871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c005180 0x7f092c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09440d0860 con 0x7f092c005180 2023-11-24T06:15:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c00c0b0 con 0x7f092c0047c0 2023-11-24T06:15:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c00c270 con 0x7f092c003b00 2023-11-24T06:15:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7fc451aa6640 1 -- 172.21.15.5:0/2902186267 shutdown_connections 2023-11-24T06:15:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7fc451aa6640 1 --2- 172.21.15.5:0/2902186267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c0d37b0 0x7fc44c0b6760 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7fc451aa6640 1 --2- 172.21.15.5:0/2902186267 >> v2:172.21.15.6:3300/0 conn(0x7fc44c0d2740 0x7fc44c0d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7f5a1554b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a100704b0 0x7f5a10098500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7f5a1554b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a100704b0 0x7f5a10098500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35746/0 (socket says 172.21.15.5:35746) 2023-11-24T06:15:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7f5a1554b640 1 -- 172.21.15.5:0/34661737 learned_addr learned my addr 172.21.15.5:0/34661737 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7f5a1654d640 1 --2- 172.21.15.5:0/34661737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a10153af0 0x7f5a100ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7fc451aa6640 1 --2- 172.21.15.5:0/2902186267 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc44c15d770 0x7fc44c0d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7fc451aa6640 1 -- 172.21.15.5:0/2902186267 >> 172.21.15.5:0/2902186267 conn(0x7fc44c0721a0 msgr2=0x7fc44c06b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:58.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7f094259c640 1 --2- 172.21.15.5:0/563432871 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f092c003b00 0x7f092c003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7fc451aa6640 1 -- 172.21.15.5:0/2902186267 shutdown_connections 2023-11-24T06:15:58.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7f0941d9b640 1 --2- 172.21.15.5:0/563432871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f092c0047c0 0x7f092c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7f5a15d4c640 1 --2- 172.21.15.5:0/34661737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a1006d520 0x7f5a1014be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7fd95e7fc640 1 -- 172.21.15.5:0/2600666789 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd954005ad0 con 0x7fd9480047c0 2023-11-24T06:15:58.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7fc451aa6640 1 -- 172.21.15.5:0/2902186267 wait complete. 2023-11-24T06:15:58.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7fd95e7fc640 1 -- 172.21.15.5:0/2600666789 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd954005c50 con 0x7fd9480047c0 2023-11-24T06:15:58.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.945+0000 7f0942d9d640 1 --2- 172.21.15.5:0/563432871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c005180 0x7f092c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7fb0bdc00640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb0b815d770 0x7fb0b81608b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.943+0000 7fb0b77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0b80d37b0 0x7fb0b81f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.945+0000 7fb0bdc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b8155b60 0x7fb0b8156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.945+0000 7fb0bdc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b8156a70 con 0x7fb0b8155b60 2023-11-24T06:15:58.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.945+0000 7fb0bdc00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b8156d50 con 0x7fb0b80d37b0 2023-11-24T06:15:58.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.945+0000 7fb0b77fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0b80d37b0 0x7fb0b81f31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:50022/0 (socket says 172.21.15.5:50022) 2023-11-24T06:15:58.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.945+0000 7fb0b77fe640 1 -- 172.21.15.5:0/84734426 learned_addr learned my addr 172.21.15.5:0/84734426 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:58.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.945+0000 7fb0bdc00640 1 -- 172.21.15.5:0/84734426 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b8157030 con 0x7fb0b815d770 2023-11-24T06:15:58.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.945+0000 7fd965bb0640 1 -- 172.21.15.5:0/2600666789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9480047c0 msgr2=0x7fd948004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.945+0000 7fd965bb0640 1 --2- 172.21.15.5:0/2600666789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9480047c0 0x7fd948004c40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd9600ca360 tx=0x7fd9480057f0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f0941d9b640 1 -- 172.21.15.5:0/563432871 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f092c003b00 msgr2=0x7f092c003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7fd965bb0640 1 -- 172.21.15.5:0/2600666789 shutdown_connections 2023-11-24T06:15:58.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7fd965bb0640 1 --2- 172.21.15.5:0/2600666789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9480047c0 0x7fd948004c40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7fd965bb0640 1 -- 172.21.15.5:0/2600666789 >> 172.21.15.5:0/2600666789 conn(0x7fd9600721a0 msgr2=0x7fd96006b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:58.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f0941d9b640 1 --2- 172.21.15.5:0/563432871 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f092c003b00 0x7f092c003f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f0941d9b640 1 -- 172.21.15.5:0/563432871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c005180 msgr2=0x7f092c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:58.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7fd965bb0640 1 -- 172.21.15.5:0/2600666789 shutdown_connections 2023-11-24T06:15:58.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7fd965bb0640 1 -- 172.21.15.5:0/2600666789 wait complete. 2023-11-24T06:15:58.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f0941d9b640 1 --2- 172.21.15.5:0/563432871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c005180 0x7f092c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f0941d9b640 1 -- 172.21.15.5:0/563432871 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09440d1720 con 0x7f092c0047c0 2023-11-24T06:15:58.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f5a1654d640 1 -- 172.21.15.5:0/34661737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a1006d520 msgr2=0x7f5a1014be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.944+0000 7f2f76ffd640 1 -- 172.21.15.5:0/1155464204 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f7c02f3d0 con 0x7f2f800d3980 2023-11-24T06:15:58.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f2f76ffd640 1 --2- 172.21.15.5:0/1155464204 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f6406af10 0x7f2f6406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f2f76ffd640 1 -- 172.21.15.5:0/1155464204 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f7c0ac100 con 0x7f2f800d3980 2023-11-24T06:15:58.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f2f76ffd640 1 --2- 172.21.15.5:0/1155464204 >> [v2:172.21.15.5:6824/1782469371,v1:172.21.15.5:6825/1782469371] conn(0x7f2f64073130 0x7f2f64075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f2f76ffd640 1 -- 172.21.15.5:0/1155464204 --> [v2:172.21.15.5:6824/1782469371,v1:172.21.15.5:6825/1782469371] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f64075c00 con 0x7f2f64073130 2023-11-24T06:15:58.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f2f76ffd640 1 -- 172.21.15.5:0/1155464204 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f7c0ac4b0 con 0x7f2f800d3980 2023-11-24T06:15:58.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f2f85d03640 1 --2- 172.21.15.5:0/1155464204 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f6406af10 0x7f2f6406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7fb0b77fe640 1 -- 172.21.15.5:0/84734426 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb0b815d770 msgr2=0x7fb0b81608b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:15:58.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7fb0b77fe640 1 --2- 172.21.15.5:0/84734426 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb0b815d770 0x7fb0b81608b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7fb0b77fe640 1 -- 172.21.15.5:0/84734426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b8155b60 msgr2=0x7fb0b8156010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T06:15:58.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.947+0000 7fb0b77fe640 1 --2- 172.21.15.5:0/84734426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b8155b60 0x7fb0b8156010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.947+0000 7fb0b77fe640 1 -- 172.21.15.5:0/84734426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a80094d0 con 0x7fb0b80d37b0 2023-11-24T06:15:58.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.947+0000 7f2f86d05640 1 --2- 172.21.15.5:0/1155464204 >> [v2:172.21.15.5:6824/1782469371,v1:172.21.15.5:6825/1782469371] conn(0x7f2f64073130 0x7f2f64075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f5a1654d640 1 --2- 172.21.15.5:0/34661737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a1006d520 0x7f5a1014be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f5a1654d640 1 -- 172.21.15.5:0/34661737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a100704b0 msgr2=0x7f5a10098500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.947+0000 7fc451aa6640 1 Processor -- start 2023-11-24T06:15:58.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.948+0000 7f2f85d03640 1 --2- 172.21.15.5:0/1155464204 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f6406af10 0x7f2f6406d3d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f80147bc0 tx=0x7f2f70007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.947+0000 7fb0b77fe640 1 --2- 172.21.15.5:0/84734426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0b80d37b0 0x7fb0b81f31d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000efc0 tx=0x7fb0b000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.948+0000 7fc451aa6640 1 -- start start 2023-11-24T06:15:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.946+0000 7f5a1654d640 1 --2- 172.21.15.5:0/34661737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a100704b0 0x7f5a10098500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.948+0000 7f0942d9d640 1 --2- 172.21.15.5:0/563432871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c005180 0x7f092c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:15:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.948+0000 7f0942d9d640 1 -- 172.21.15.5:0/563432871 reap_dead start 2023-11-24T06:15:58.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.948+0000 7fb0affff640 1 -- 172.21.15.5:0/84734426 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0014480 con 0x7fb0b80d37b0 2023-11-24T06:15:58.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.948+0000 7fb0affff640 1 -- 172.21.15.5:0/84734426 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b000f040 con 0x7fb0b80d37b0 2023-11-24T06:15:58.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.948+0000 7fb0bdc00640 1 -- 172.21.15.5:0/84734426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b8157310 con 0x7fb0b80d37b0 2023-11-24T06:15:58.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.949+0000 7f2f86d05640 1 --2- 172.21.15.5:0/1155464204 >> [v2:172.21.15.5:6824/1782469371,v1:172.21.15.5:6825/1782469371] conn(0x7f2f64073130 0x7f2f64075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.949+0000 7f5a1654d640 1 -- 172.21.15.5:0/34661737 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a08005d50 con 0x7f5a10153af0 2023-11-24T06:15:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.948+0000 7fb0affff640 1 -- 172.21.15.5:0/84734426 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00147e0 con 0x7fb0b80d37b0 2023-11-24T06:15:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.948+0000 7fb0bdc00640 1 -- 172.21.15.5:0/84734426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b81f3710 con 0x7fb0b80d37b0 2023-11-24T06:15:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.948+0000 7f0941d9b640 1 --2- 172.21.15.5:0/563432871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f092c0047c0 0x7f092c004c40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09440ca360 tx=0x7f092c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=202aecde439eb677 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.949+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938005ad0 con 0x7f092c0047c0 2023-11-24T06:15:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.949+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0938005c30 con 0x7f092c0047c0 2023-11-24T06:15:58.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.949+0000 7f094159a640 1 -- 172.21.15.5:0/563432871 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093802c850 con 0x7f092c0047c0 2023-11-24T06:15:58.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.950+0000 7fb0bdc00640 1 -- 172.21.15.5:0/84734426 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb0b815def0 con 0x7fb0b80d37b0 2023-11-24T06:15:58.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.950+0000 7fd965bb0640 1 Processor -- start 2023-11-24T06:15:58.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.950+0000 7f0948e56640 1 -- 172.21.15.5:0/563432871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f092c0047c0 msgr2=0x7f092c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.950+0000 7f0948e56640 1 --2- 172.21.15.5:0/563432871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f092c0047c0 0x7f092c004c40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09440ca360 tx=0x7f092c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.950+0000 7f2f76ffd640 1 -- 172.21.15.5:0/1155464204 <== osd.2 v2:172.21.15.5:6824/1782469371 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f2f64075c00 con 0x7f2f64073130 2023-11-24T06:15:58.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.950+0000 7f5a1654d640 1 --2- 172.21.15.5:0/34661737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a10153af0 0x7f5a100ce330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c3f0 tx=0x7f5a0000c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:58.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.950+0000 7fc451aa6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc44c0d37b0 0x7fc44c1ff930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.950+0000 7f0948e56640 1 -- 172.21.15.5:0/563432871 shutdown_connections 2023-11-24T06:15:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.951+0000 7f0948e56640 1 --2- 172.21.15.5:0/563432871 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f092c0047c0 0x7f092c004c40 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.951+0000 7f0948e56640 1 -- 172.21.15.5:0/563432871 >> 172.21.15.5:0/563432871 conn(0x7f09440721a0 msgr2=0x7f094406b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.950+0000 7fd965bb0640 1 -- start start 2023-11-24T06:15:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.951+0000 7f0948e56640 1 -- 172.21.15.5:0/563432871 shutdown_connections 2023-11-24T06:15:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.951+0000 7f5a067fc640 1 -- 172.21.15.5:0/34661737 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00005ce0 con 0x7f5a10153af0 2023-11-24T06:15:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.951+0000 7f5a067fc640 1 -- 172.21.15.5:0/34661737 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a00005e40 con 0x7f5a10153af0 2023-11-24T06:15:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.951+0000 7f5a067fc640 1 -- 172.21.15.5:0/34661737 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a000072d0 con 0x7f5a10153af0 2023-11-24T06:15:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.951+0000 7fc451aa6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c15d770 0x7fc44c154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.951+0000 7fc451aa6640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc44c154e00 0x7fc44c1547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.952+0000 7fc451aa6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c15c0a0 con 0x7fc44c15d770 2023-11-24T06:15:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.952+0000 7fc44b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc44c0d37b0 0x7fc44c1ff930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.952+0000 7fc451aa6640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c15c380 con 0x7fc44c0d37b0 2023-11-24T06:15:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.952+0000 7fc451aa6640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c15c660 con 0x7fc44c154e00 2023-11-24T06:15:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.951+0000 7f5a17fb0640 1 -- 172.21.15.5:0/34661737 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a101ee920 con 0x7f5a10153af0 2023-11-24T06:15:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.952+0000 7f5a17fb0640 1 -- 172.21.15.5:0/34661737 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a100b3590 con 0x7f5a10153af0 2023-11-24T06:15:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.952+0000 7fd965bb0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9480047c0 0x7fd96014f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.952+0000 7fd965bb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9600cb400 0x7fd9600cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.952+0000 7fd965bb0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd960154720 0x7fd960154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.952+0000 7fd965bb0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9600cbdc0 con 0x7fd9600cb400 2023-11-24T06:15:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.952+0000 7fd965bb0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9601560a0 con 0x7fd9480047c0 2023-11-24T06:15:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.952+0000 7fd965bb0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd960155490 con 0x7fd960154720 2023-11-24T06:15:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.952+0000 7fc44b7fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc44c0d37b0 0x7fc44c1ff930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:50052/0 (socket says 172.21.15.5:50052) 2023-11-24T06:15:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.952+0000 7fc44b7fe640 1 -- 172.21.15.5:0/3289771613 learned_addr learned my addr 172.21.15.5:0/3289771613 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.953+0000 7fb0affff640 1 -- 172.21.15.5:0/84734426 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb0b0014940 con 0x7fb0b80d37b0 2023-11-24T06:15:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.953+0000 7fc44bfff640 1 --2- 172.21.15.5:0/3289771613 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc44c154e00 0x7fc44c1547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.953+0000 7fd95ffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd960154720 0x7fd960154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.951+0000 7f0948e56640 1 -- 172.21.15.5:0/563432871 wait complete. 2023-11-24T06:15:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.953+0000 7fd95ffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd960154720 0x7fd960154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:53362/0 (socket says 172.21.15.5:53362) 2023-11-24T06:15:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.953+0000 7fd95ffff640 1 -- 172.21.15.5:0/1626614781 learned_addr learned my addr 172.21.15.5:0/1626614781 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.953+0000 7fc44b7fe640 1 -- 172.21.15.5:0/3289771613 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc44c154e00 msgr2=0x7fc44c1547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.953+0000 7fc44b7fe640 1 --2- 172.21.15.5:0/3289771613 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc44c154e00 0x7fc44c1547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.953+0000 7fc44b7fe640 1 -- 172.21.15.5:0/3289771613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c15d770 msgr2=0x7fc44c154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.953+0000 7fc44b7fe640 1 --2- 172.21.15.5:0/3289771613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc44c15d770 0x7fc44c154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.953+0000 7fc44b7fe640 1 -- 172.21.15.5:0/3289771613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc43c009400 con 0x7fc44c0d37b0 2023-11-24T06:15:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.953+0000 7f0948e56640 1 Processor -- start 2023-11-24T06:15:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.954+0000 7f0948e56640 1 -- start start 2023-11-24T06:15:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.954+0000 7fd95effd640 1 --2- 172.21.15.5:0/1626614781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9600cb400 0x7fd9600cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.954+0000 7fd95ffff640 1 -- 172.21.15.5:0/1626614781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9480047c0 msgr2=0x7fd96014f6d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:15:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.954+0000 7f5a17fb0640 1 -- 172.21.15.5:0/34661737 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5a1015a1a0 con 0x7f5a10153af0 2023-11-24T06:15:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.954+0000 7fb0affff640 1 --2- 172.21.15.5:0/84734426 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb09806ae40 0x7fb09806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.955+0000 7fb0affff640 1 -- 172.21.15.5:0/84734426 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0b008ace0 con 0x7fb0b80d37b0 2023-11-24T06:15:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.955+0000 7fb0affff640 1 --2- 172.21.15.5:0/84734426 >> [v2:172.21.15.6:6822/2056352444,v1:172.21.15.6:6823/2056352444] conn(0x7fb098073060 0x7fb098075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.955+0000 7fb0affff640 1 -- 172.21.15.5:0/84734426 --> [v2:172.21.15.6:6822/2056352444,v1:172.21.15.6:6823/2056352444] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb098075b30 con 0x7fb098073060 2023-11-24T06:15:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.955+0000 7fb0affff640 1 -- 172.21.15.5:0/84734426 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0b008b090 con 0x7fb0b80d37b0 2023-11-24T06:15:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.955+0000 7fb0b6ffd640 1 --2- 172.21.15.5:0/84734426 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb09806ae40 0x7fb09806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.955+0000 7fc44b7fe640 1 --2- 172.21.15.5:0/3289771613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc44c0d37b0 0x7fc44c1ff930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc43400b410 tx=0x7fc43400b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.955+0000 7fc448ff9640 1 -- 172.21.15.5:0/3289771613 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434011830 con 0x7fc44c0d37b0 2023-11-24T06:15:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.955+0000 7fc448ff9640 1 -- 172.21.15.5:0/3289771613 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4340119b0 con 0x7fc44c0d37b0 2023-11-24T06:15:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.955+0000 7fc448ff9640 1 -- 172.21.15.5:0/3289771613 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434011d60 con 0x7fc44c0d37b0 2023-11-24T06:15:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.955+0000 7f0948e56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c0047c0 0x7f09440ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.955+0000 7f0948e56640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09441f5fd0 0x7f09441f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.955+0000 7f0948e56640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09440cc460 0x7f09440cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.954+0000 7fd95ffff640 1 --2- 172.21.15.5:0/1626614781 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd9480047c0 0x7fd96014f6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.954+0000 7fd95ffff640 1 -- 172.21.15.5:0/1626614781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9600cb400 msgr2=0x7fd9600cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.955+0000 7fc451aa6640 1 -- 172.21.15.5:0/3289771613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc444009b30 con 0x7fc44c0d37b0 2023-11-24T06:15:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.956+0000 7fc451aa6640 1 -- 172.21.15.5:0/3289771613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c15cc20 con 0x7fc44c0d37b0 2023-11-24T06:15:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.956+0000 7fb0b7fff640 1 --2- 172.21.15.5:0/84734426 >> [v2:172.21.15.6:6822/2056352444,v1:172.21.15.6:6823/2056352444] conn(0x7fb098073060 0x7fb098075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.956+0000 7f0948e56640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0944150e30 con 0x7f092c0047c0 2023-11-24T06:15:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.956+0000 7f0948e56640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09440cce20 con 0x7f09440cc460 2023-11-24T06:15:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.956+0000 7f0948e56640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0944150b50 con 0x7f09441f5fd0 2023-11-24T06:15:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.957+0000 7f0941d9b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09441f5fd0 0x7f09441f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.957+0000 7f0941d9b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09441f5fd0 0x7f09441f6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:53370/0 (socket says 172.21.15.5:53370) 2023-11-24T06:15:59.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.957+0000 7f0941d9b640 1 -- 172.21.15.5:0/1488523483 learned_addr learned my addr 172.21.15.5:0/1488523483 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:59.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.957+0000 7f0942d9d640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09440cc460 0x7f09440cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.956+0000 7fd95ffff640 1 --2- 172.21.15.5:0/1626614781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9600cb400 0x7fd9600cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.956+0000 7fd95ffff640 1 -- 172.21.15.5:0/1626614781 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd960155770 con 0x7fd960154720 2023-11-24T06:15:59.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.956+0000 7fd95effd640 1 --2- 172.21.15.5:0/1626614781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9600cb400 0x7fd9600cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:15:59.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.957+0000 7fc451aa6640 1 -- 172.21.15.5:0/3289771613 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc44c15c4f0 con 0x7fc44c0d37b0 2023-11-24T06:15:59.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.957+0000 7f5a067fc640 1 -- 172.21.15.5:0/34661737 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a00007430 con 0x7f5a10153af0 2023-11-24T06:15:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.957+0000 7f094259c640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c0047c0 0x7f09440ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.957+0000 7f0941d9b640 1 -- 172.21.15.5:0/1488523483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09440cc460 msgr2=0x7f09440cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.957+0000 7f0941d9b640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09440cc460 0x7f09440cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.957+0000 7f0941d9b640 1 -- 172.21.15.5:0/1488523483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c0047c0 msgr2=0x7f09440ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.957+0000 7f0941d9b640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c0047c0 0x7f09440ca3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.957+0000 7f0941d9b640 1 -- 172.21.15.5:0/1488523483 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094415eed0 con 0x7f09441f5fd0 2023-11-24T06:15:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.958+0000 7f094259c640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f092c0047c0 0x7f09440ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:15:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.958+0000 7f0941d9b640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09441f5fd0 0x7f09441f6430 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0938001d80 tx=0x7f0938001930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.958+0000 7fd95ffff640 1 --2- 172.21.15.5:0/1626614781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd960154720 0x7fd960154ba0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd950007fd0 tx=0x7fd95000c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.959+0000 7f5a067fc640 1 --2- 172.21.15.5:0/34661737 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f59f406ae40 0x7f59f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.959+0000 7f5a067fc640 1 -- 172.21.15.5:0/34661737 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a00088c40 con 0x7f5a10153af0 2023-11-24T06:15:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.959+0000 7f5a067fc640 1 --2- 172.21.15.5:0/34661737 >> [v2:172.21.15.5:6808/337602632,v1:172.21.15.5:6809/337602632] conn(0x7f59f4073060 0x7f59f4075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.959+0000 7fd947fff640 1 -- 172.21.15.5:0/1626614781 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9500070d0 con 0x7fd960154720 2023-11-24T06:15:59.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.959+0000 7fd947fff640 1 -- 172.21.15.5:0/1626614781 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd950007230 con 0x7fd960154720 2023-11-24T06:15:59.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.959+0000 7fd947fff640 1 -- 172.21.15.5:0/1626614781 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9500075e0 con 0x7fd960154720 2023-11-24T06:15:59.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.959+0000 7fb0b7fff640 1 --2- 172.21.15.5:0/84734426 >> [v2:172.21.15.6:6822/2056352444,v1:172.21.15.6:6823/2056352444] conn(0x7fb098073060 0x7fb098075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.960+0000 7fb0b6ffd640 1 --2- 172.21.15.5:0/84734426 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb09806ae40 0x7fb09806d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8036000 tx=0x7fb0a8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.959+0000 7f0932ffd640 1 -- 172.21.15.5:0/1488523483 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093803c070 con 0x7f09441f5fd0 2023-11-24T06:15:59.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.959+0000 7f0932ffd640 1 -- 172.21.15.5:0/1488523483 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093802ca60 con 0x7f09441f5fd0 2023-11-24T06:15:59.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.959+0000 7f0932ffd640 1 -- 172.21.15.5:0/1488523483 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093802cdd0 con 0x7f09441f5fd0 2023-11-24T06:15:59.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.959+0000 7f0948e56640 1 -- 172.21.15.5:0/1488523483 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f094415f1b0 con 0x7f09441f5fd0 2023-11-24T06:15:59.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.960+0000 7f0948e56640 1 -- 172.21.15.5:0/1488523483 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094414fe00 con 0x7f09441f5fd0 2023-11-24T06:15:59.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.959+0000 7f5a067fc640 1 -- 172.21.15.5:0/34661737 --> [v2:172.21.15.5:6808/337602632,v1:172.21.15.5:6809/337602632] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59f4075b30 con 0x7f59f4073060 2023-11-24T06:15:59.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.959+0000 7f5a067fc640 1 -- 172.21.15.5:0/34661737 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a00036ea0 con 0x7f5a10153af0 2023-11-24T06:15:59.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.960+0000 7f5a15d4c640 1 --2- 172.21.15.5:0/34661737 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f59f406ae40 0x7f59f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.960+0000 7fd965bb0640 1 -- 172.21.15.5:0/1626614781 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd960155a50 con 0x7fd960154720 2023-11-24T06:15:59.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.960+0000 7fd965bb0640 1 -- 172.21.15.5:0/1626614781 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96014fc10 con 0x7fd960154720 2023-11-24T06:15:59.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.960+0000 7f5a1554b640 1 --2- 172.21.15.5:0/34661737 >> [v2:172.21.15.5:6808/337602632,v1:172.21.15.5:6809/337602632] conn(0x7f59f4073060 0x7f59f4075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.961+0000 7f0948e56640 1 -- 172.21.15.5:0/1488523483 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09440d1ce0 con 0x7f09441f5fd0 2023-11-24T06:15:59.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.961+0000 7f5a1554b640 1 --2- 172.21.15.5:0/34661737 >> [v2:172.21.15.5:6808/337602632,v1:172.21.15.5:6809/337602632] conn(0x7f59f4073060 0x7f59f4075480 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.961+0000 7f5a15d4c640 1 --2- 172.21.15.5:0/34661737 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f59f406ae40 0x7f59f406d300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5a08038000 tx=0x7f5a08032000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.962+0000 7fb0affff640 1 -- 172.21.15.5:0/84734426 <== osd.4 v2:172.21.15.6:6822/2056352444 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb098075b30 con 0x7fb098073060 2023-11-24T06:15:59.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.962+0000 7fd965bb0640 1 -- 172.21.15.5:0/1626614781 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9600d1ce0 con 0x7fd960154720 2023-11-24T06:15:59.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.964+0000 7fc448ff9640 1 -- 172.21.15.5:0/3289771613 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc43401a400 con 0x7fc44c0d37b0 2023-11-24T06:15:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.963+0000 7f5a067fc640 1 -- 172.21.15.5:0/34661737 <== osd.3 v2:172.21.15.5:6808/337602632 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f59f4075b30 con 0x7f59f4073060 2023-11-24T06:15:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.963+0000 7f0932ffd640 1 -- 172.21.15.5:0/1488523483 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0938032070 con 0x7f09441f5fd0 2023-11-24T06:15:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.964+0000 7f0932ffd640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f092406ae40 0x7f092406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.964+0000 7f0932ffd640 1 -- 172.21.15.5:0/1488523483 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09380aad60 con 0x7f09441f5fd0 2023-11-24T06:15:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.965+0000 7f0932ffd640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.6:6800/3179572934,v1:172.21.15.6:6801/3179572934] conn(0x7f0924073060 0x7f0924075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.965+0000 7f0932ffd640 1 -- 172.21.15.5:0/1488523483 --> [v2:172.21.15.6:6800/3179572934,v1:172.21.15.6:6801/3179572934] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0924075b30 con 0x7f0924073060 2023-11-24T06:15:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.965+0000 7f0932ffd640 1 -- 172.21.15.5:0/1488523483 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09380ab110 con 0x7f09441f5fd0 2023-11-24T06:15:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.965+0000 7f094259c640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f092406ae40 0x7f092406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.966+0000 7f0942d9d640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.6:6800/3179572934,v1:172.21.15.6:6801/3179572934] conn(0x7f0924073060 0x7f0924075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.967+0000 7f094259c640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f092406ae40 0x7f092406d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0944068d90 tx=0x7f0934008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.968+0000 7f0942d9d640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.6:6800/3179572934,v1:172.21.15.6:6801/3179572934] conn(0x7f0924073060 0x7f0924075480 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.965+0000 7fd947fff640 1 -- 172.21.15.5:0/1626614781 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd950007740 con 0x7fd960154720 2023-11-24T06:15:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.967+0000 7fd947fff640 1 --2- 172.21.15.5:0/1626614781 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd94c06ae90 0x7fd94c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.967+0000 7fd947fff640 1 -- 172.21.15.5:0/1626614781 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd950058640 con 0x7fd960154720 2023-11-24T06:15:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.967+0000 7fd95f7fe640 1 --2- 172.21.15.5:0/1626614781 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd94c06ae90 0x7fd94c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.968+0000 7fd95f7fe640 1 --2- 172.21.15.5:0/1626614781 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd94c06ae90 0x7fd94c06d350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd958009a40 tx=0x7fd958009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.965+0000 7fc448ff9640 1 --2- 172.21.15.5:0/3289771613 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc42c06ae90 0x7fc42c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.966+0000 7fc448ff9640 1 -- 172.21.15.5:0/3289771613 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc434089cf0 con 0x7fc44c0d37b0 2023-11-24T06:15:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.966+0000 7fc448ff9640 1 --2- 172.21.15.5:0/3289771613 >> [v2:172.21.15.6:6808/4073132861,v1:172.21.15.6:6809/4073132861] conn(0x7fc42c0730b0 0x7fc42c0754d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.966+0000 7fc448ff9640 1 -- 172.21.15.5:0/3289771613 --> [v2:172.21.15.6:6808/4073132861,v1:172.21.15.6:6809/4073132861] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc42c075b80 con 0x7fc42c0730b0 2023-11-24T06:15:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.966+0000 7fc448ff9640 1 -- 172.21.15.5:0/3289771613 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc43408a110 con 0x7fc44c0d37b0 2023-11-24T06:15:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.966+0000 7fc443fff640 1 --2- 172.21.15.5:0/3289771613 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc42c06ae90 0x7fc42c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.968+0000 7fc44bfff640 1 --2- 172.21.15.5:0/3289771613 >> [v2:172.21.15.6:6808/4073132861,v1:172.21.15.6:6809/4073132861] conn(0x7fc42c0730b0 0x7fc42c0754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.968+0000 7fc443fff640 1 --2- 172.21.15.5:0/3289771613 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc42c06ae90 0x7fc42c06d350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc43c006e80 tx=0x7fc43c006dd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.969+0000 7fd947fff640 1 --2- 172.21.15.5:0/1626614781 >> [v2:172.21.15.6:6814/2361559103,v1:172.21.15.6:6816/2361559103] conn(0x7fd94c073d90 0x7fd94c076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.969+0000 7fd947fff640 1 -- 172.21.15.5:0/1626614781 --> [v2:172.21.15.6:6814/2361559103,v1:172.21.15.6:6816/2361559103] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd94c076900 con 0x7fd94c073d90 2023-11-24T06:15:59.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.969+0000 7fd947fff640 1 -- 172.21.15.5:0/1626614781 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd950003c40 con 0x7fd960154720 2023-11-24T06:15:59.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.969+0000 7fc44bfff640 1 --2- 172.21.15.5:0/3289771613 >> [v2:172.21.15.6:6808/4073132861,v1:172.21.15.6:6809/4073132861] conn(0x7fc42c0730b0 0x7fc42c0754d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.969+0000 7fd95effd640 1 --2- 172.21.15.5:0/1626614781 >> [v2:172.21.15.6:6814/2361559103,v1:172.21.15.6:6816/2361559103] conn(0x7fd94c073d90 0x7fd94c076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.970+0000 7f0932ffd640 1 -- 172.21.15.5:0/1488523483 <== osd.5 v2:172.21.15.6:6800/3179572934 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0924075b30 con 0x7f0924073060 2023-11-24T06:15:59.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.970+0000 7fd95effd640 1 --2- 172.21.15.5:0/1626614781 >> [v2:172.21.15.6:6814/2361559103,v1:172.21.15.6:6816/2361559103] conn(0x7fd94c073d90 0x7fd94c076230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.972+0000 7f0548d0c640 1 -- 172.21.15.5:0/2807622637 --> [v2:172.21.15.5:6816/1571096651,v1:172.21.15.5:6817/1571096651] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f05440726a0 con 0x7f0524073060 2023-11-24T06:15:59.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.972+0000 7fc448ff9640 1 -- 172.21.15.5:0/3289771613 <== osd.7 v2:172.21.15.6:6808/4073132861 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc42c075b80 con 0x7fc42c0730b0 2023-11-24T06:15:59.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.972+0000 7fe010c6b640 1 -- 172.21.15.5:0/194067898 --> [v2:172.21.15.5:6800/1318127149,v1:172.21.15.5:6801/1318127149] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdff0002cb0 con 0x7fdfec073090 2023-11-24T06:15:59.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.973+0000 7fd947fff640 1 -- 172.21.15.5:0/1626614781 <== osd.6 v2:172.21.15.6:6814/2361559103 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd94c076900 con 0x7fd94c073d90 2023-11-24T06:15:59.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.973+0000 7fe002ffd640 1 -- 172.21.15.5:0/194067898 <== osd.1 v2:172.21.15.5:6800/1318127149 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdff0002cb0 con 0x7fdfec073090 2023-11-24T06:15:59.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.973+0000 7f0532ffd640 1 -- 172.21.15.5:0/2807622637 <== osd.0 v2:172.21.15.5:6816/1571096651 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0538009870 con 0x7f0524073060 2023-11-24T06:15:59.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.974+0000 7f0548d0c640 1 -- 172.21.15.5:0/2807622637 >> [v2:172.21.15.5:6816/1571096651,v1:172.21.15.5:6817/1571096651] conn(0x7f0524073060 msgr2=0x7f0524075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.974+0000 7f0548d0c640 1 --2- 172.21.15.5:0/2807622637 >> [v2:172.21.15.5:6816/1571096651,v1:172.21.15.5:6817/1571096651] conn(0x7f0524073060 0x7f0524075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.974+0000 7f0548d0c640 1 -- 172.21.15.5:0/2807622637 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f052406ae40 msgr2=0x7f052406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.974+0000 7f0548d0c640 1 --2- 172.21.15.5:0/2807622637 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f052406ae40 0x7f052406d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0534036000 tx=0x7f0534030000 comp rx=0 tx=0).stop 2023-11-24T06:15:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.974+0000 7f0548d0c640 1 -- 172.21.15.5:0/2807622637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05440b3cb0 msgr2=0x7f05440cd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.974+0000 7f0548d0c640 1 --2- 172.21.15.5:0/2807622637 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f05440b3cb0 0x7f05440cd120 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f053c00ba50 tx=0x7f053c00bf10 comp rx=0 tx=0).stop 2023-11-24T06:15:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.974+0000 7f0542d9d640 1 -- 172.21.15.5:0/2807622637 reap_dead start 2023-11-24T06:15:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.975+0000 7f0548d0c640 1 -- 172.21.15.5:0/2807622637 shutdown_connections 2023-11-24T06:15:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.975+0000 7f0548d0c640 1 -- 172.21.15.5:0/2807622637 >> 172.21.15.5:0/2807622637 conn(0x7f05440709f0 msgr2=0x7f054416dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.977+0000 7fe010c6b640 1 -- 172.21.15.5:0/194067898 >> [v2:172.21.15.5:6800/1318127149,v1:172.21.15.5:6801/1318127149] conn(0x7fdfec073090 msgr2=0x7fdfec0754b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.977+0000 7fe010c6b640 1 --2- 172.21.15.5:0/194067898 >> [v2:172.21.15.5:6800/1318127149,v1:172.21.15.5:6801/1318127149] conn(0x7fdfec073090 0x7fdfec0754b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.977+0000 7f0548d0c640 1 -- 172.21.15.5:0/2807622637 shutdown_connections 2023-11-24T06:15:59.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.977+0000 7fe010c6b640 1 -- 172.21.15.5:0/194067898 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdfec06ae70 msgr2=0x7fdfec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.977+0000 7fe010c6b640 1 --2- 172.21.15.5:0/194067898 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdfec06ae70 0x7fdfec06d330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdff4036000 tx=0x7fdff403c040 comp rx=0 tx=0).stop 2023-11-24T06:15:59.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.977+0000 7fe010c6b640 1 -- 172.21.15.5:0/194067898 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe00c071fb0 msgr2=0x7fe00c14e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.977+0000 7fe010c6b640 1 --2- 172.21.15.5:0/194067898 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe00c071fb0 0x7fe00c14e560 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00c850 tx=0x7fdffc00cd10 comp rx=0 tx=0).stop 2023-11-24T06:15:59.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.977+0000 7f0548d0c640 1 -- 172.21.15.5:0/2807622637 wait complete. 2023-11-24T06:15:59.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.980+0000 7fe00ad9d640 1 -- 172.21.15.5:0/194067898 reap_dead start 2023-11-24T06:15:59.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.980+0000 7fe010c6b640 1 -- 172.21.15.5:0/194067898 shutdown_connections 2023-11-24T06:15:59.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.980+0000 7fe010c6b640 1 -- 172.21.15.5:0/194067898 >> 172.21.15.5:0/194067898 conn(0x7fe00c0709f0 msgr2=0x7fe00c152eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.980+0000 7fe010c6b640 1 -- 172.21.15.5:0/194067898 shutdown_connections 2023-11-24T06:15:59.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.980+0000 7fe010c6b640 1 -- 172.21.15.5:0/194067898 wait complete. 2023-11-24T06:15:59.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.985+0000 7f2f88768640 1 -- 172.21.15.5:0/1155464204 --> [v2:172.21.15.5:6824/1782469371,v1:172.21.15.5:6825/1782469371] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2f800d1ce0 con 0x7f2f64073130 2023-11-24T06:15:59.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.987+0000 7f2f76ffd640 1 -- 172.21.15.5:0/1155464204 <== osd.2 v2:172.21.15.5:6824/1782469371 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2f7800f040 con 0x7f2f64073130 2023-11-24T06:15:59.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.987+0000 7f2f88768640 1 -- 172.21.15.5:0/1155464204 >> [v2:172.21.15.5:6824/1782469371,v1:172.21.15.5:6825/1782469371] conn(0x7f2f64073130 msgr2=0x7f2f64075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.987+0000 7f5a17fb0640 1 -- 172.21.15.5:0/34661737 --> [v2:172.21.15.5:6808/337602632,v1:172.21.15.5:6809/337602632] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5a10159ec0 con 0x7f59f4073060 2023-11-24T06:15:59.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.988+0000 7f2f88768640 1 --2- 172.21.15.5:0/1155464204 >> [v2:172.21.15.5:6824/1782469371,v1:172.21.15.5:6825/1782469371] conn(0x7f2f64073130 0x7f2f64075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.988+0000 7f2f88768640 1 -- 172.21.15.5:0/1155464204 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f6406af10 msgr2=0x7f2f6406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.988+0000 7f2f88768640 1 --2- 172.21.15.5:0/1155464204 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f6406af10 0x7f2f6406d3d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f80147bc0 tx=0x7f2f70007b50 comp rx=0 tx=0).stop 2023-11-24T06:15:59.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.988+0000 7f2f88768640 1 -- 172.21.15.5:0/1155464204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f800d3980 msgr2=0x7f2f800cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.988+0000 7f2f88768640 1 --2- 172.21.15.5:0/1155464204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f800d3980 0x7f2f800cbe40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c033000 tx=0x7f2f7c004730 comp rx=0 tx=0).stop 2023-11-24T06:15:59.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.988+0000 7f2f86d05640 1 -- 172.21.15.5:0/1155464204 reap_dead start 2023-11-24T06:15:59.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.988+0000 7f5a067fc640 1 -- 172.21.15.5:0/34661737 <== osd.3 v2:172.21.15.5:6808/337602632 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5a0c00b6a0 con 0x7f59f4073060 2023-11-24T06:15:59.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.989+0000 7f2f88768640 1 -- 172.21.15.5:0/1155464204 shutdown_connections 2023-11-24T06:15:59.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.989+0000 7f2f88768640 1 -- 172.21.15.5:0/1155464204 >> 172.21.15.5:0/1155464204 conn(0x7f2f800721a0 msgr2=0x7f2f800b7370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.989+0000 7f5a17fb0640 1 -- 172.21.15.5:0/34661737 >> [v2:172.21.15.5:6808/337602632,v1:172.21.15.5:6809/337602632] conn(0x7f59f4073060 msgr2=0x7f59f4075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.989+0000 7f5a17fb0640 1 --2- 172.21.15.5:0/34661737 >> [v2:172.21.15.5:6808/337602632,v1:172.21.15.5:6809/337602632] conn(0x7f59f4073060 0x7f59f4075480 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.989+0000 7f5a17fb0640 1 -- 172.21.15.5:0/34661737 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f59f406ae40 msgr2=0x7f59f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.989+0000 7f5a17fb0640 1 --2- 172.21.15.5:0/34661737 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f59f406ae40 0x7f59f406d300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5a08038000 tx=0x7f5a08032000 comp rx=0 tx=0).stop 2023-11-24T06:15:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.990+0000 7f5a17fb0640 1 -- 172.21.15.5:0/34661737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a10153af0 msgr2=0x7f5a100ce330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.990+0000 7f5a17fb0640 1 --2- 172.21.15.5:0/34661737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a10153af0 0x7f5a100ce330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c3f0 tx=0x7f5a0000c8b0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.990+0000 7f2f88768640 1 -- 172.21.15.5:0/1155464204 shutdown_connections 2023-11-24T06:15:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.990+0000 7f5a1654d640 1 -- 172.21.15.5:0/34661737 reap_dead start 2023-11-24T06:15:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.990+0000 7f2f88768640 1 -- 172.21.15.5:0/1155464204 wait complete. 2023-11-24T06:15:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.990+0000 7f5a17fb0640 1 -- 172.21.15.5:0/34661737 shutdown_connections 2023-11-24T06:15:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.990+0000 7f5a17fb0640 1 -- 172.21.15.5:0/34661737 >> 172.21.15.5:0/34661737 conn(0x7f5a100b0d80 msgr2=0x7f5a1015b760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.991+0000 7f5a17fb0640 1 -- 172.21.15.5:0/34661737 shutdown_connections 2023-11-24T06:15:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.991+0000 7f5a17fb0640 1 -- 172.21.15.5:0/34661737 wait complete. 2023-11-24T06:15:59.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.994+0000 7fb0bdc00640 1 -- 172.21.15.5:0/84734426 --> [v2:172.21.15.6:6822/2056352444,v1:172.21.15.6:6823/2056352444] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb0b80d1ce0 con 0x7fb098073060 2023-11-24T06:15:59.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.995+0000 7fb0affff640 1 -- 172.21.15.5:0/84734426 <== osd.4 v2:172.21.15.6:6822/2056352444 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb0a0003900 con 0x7fb098073060 2023-11-24T06:15:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.996+0000 7fb0bdc00640 1 -- 172.21.15.5:0/84734426 >> [v2:172.21.15.6:6822/2056352444,v1:172.21.15.6:6823/2056352444] conn(0x7fb098073060 msgr2=0x7fb098075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.996+0000 7fb0bdc00640 1 --2- 172.21.15.5:0/84734426 >> [v2:172.21.15.6:6822/2056352444,v1:172.21.15.6:6823/2056352444] conn(0x7fb098073060 0x7fb098075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.997+0000 7fb0bdc00640 1 -- 172.21.15.5:0/84734426 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb09806ae40 msgr2=0x7fb09806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.997+0000 7fb0bdc00640 1 --2- 172.21.15.5:0/84734426 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb09806ae40 0x7fb09806d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8036000 tx=0x7fb0a8008030 comp rx=0 tx=0).stop 2023-11-24T06:15:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.997+0000 7fb0bdc00640 1 -- 172.21.15.5:0/84734426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0b80d37b0 msgr2=0x7fb0b81f31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.997+0000 7fb0bdc00640 1 --2- 172.21.15.5:0/84734426 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb0b80d37b0 0x7fb0b81f31d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000efc0 tx=0x7fb0b000c490 comp rx=0 tx=0).stop 2023-11-24T06:15:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.997+0000 7fb0b7fff640 1 -- 172.21.15.5:0/84734426 reap_dead start 2023-11-24T06:15:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.997+0000 7fb0bdc00640 1 -- 172.21.15.5:0/84734426 shutdown_connections 2023-11-24T06:15:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.997+0000 7fb0bdc00640 1 -- 172.21.15.5:0/84734426 >> 172.21.15.5:0/84734426 conn(0x7fb0b80721a0 msgr2=0x7fb0b80b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.998+0000 7fb0bdc00640 1 -- 172.21.15.5:0/84734426 shutdown_connections 2023-11-24T06:15:59.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:58.998+0000 7fb0bdc00640 1 -- 172.21.15.5:0/84734426 wait complete. 2023-11-24T06:15:59.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.000+0000 7fd965bb0640 1 -- 172.21.15.5:0/1626614781 --> [v2:172.21.15.6:6814/2361559103,v1:172.21.15.6:6816/2361559103] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd9601561e0 con 0x7fd94c073d90 2023-11-24T06:15:59.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.002+0000 7fd947fff640 1 -- 172.21.15.5:0/1626614781 <== osd.6 v2:172.21.15.6:6814/2361559103 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd95403a040 con 0x7fd94c073d90 2023-11-24T06:15:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.003+0000 7fd965bb0640 1 -- 172.21.15.5:0/1626614781 >> [v2:172.21.15.6:6814/2361559103,v1:172.21.15.6:6816/2361559103] conn(0x7fd94c073d90 msgr2=0x7fd94c076230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.003+0000 7fd965bb0640 1 --2- 172.21.15.5:0/1626614781 >> [v2:172.21.15.6:6814/2361559103,v1:172.21.15.6:6816/2361559103] conn(0x7fd94c073d90 0x7fd94c076230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.003+0000 7fd965bb0640 1 -- 172.21.15.5:0/1626614781 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd94c06ae90 msgr2=0x7fd94c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.003+0000 7fd965bb0640 1 --2- 172.21.15.5:0/1626614781 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd94c06ae90 0x7fd94c06d350 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd958009a40 tx=0x7fd958009190 comp rx=0 tx=0).stop 2023-11-24T06:15:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.003+0000 7fd965bb0640 1 -- 172.21.15.5:0/1626614781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd960154720 msgr2=0x7fd960154ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.003+0000 7fd965bb0640 1 --2- 172.21.15.5:0/1626614781 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd960154720 0x7fd960154ba0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd950007fd0 tx=0x7fd95000c890 comp rx=0 tx=0).stop 2023-11-24T06:15:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.004+0000 7fd95ffff640 1 -- 172.21.15.5:0/1626614781 reap_dead start 2023-11-24T06:15:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.004+0000 7fd965bb0640 1 -- 172.21.15.5:0/1626614781 shutdown_connections 2023-11-24T06:15:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.004+0000 7fd965bb0640 1 -- 172.21.15.5:0/1626614781 >> 172.21.15.5:0/1626614781 conn(0x7fd9600721a0 msgr2=0x7fd9600cfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.029 INFO:teuthology.orchestra.run.smithi005.stdout:60129542149 2023-11-24T06:15:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.004+0000 7fd965bb0640 1 -- 172.21.15.5:0/1626614781 shutdown_connections 2023-11-24T06:15:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.004+0000 7fd965bb0640 1 -- 172.21.15.5:0/1626614781 wait complete. 2023-11-24T06:15:59.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T06:15:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.005+0000 7f0948e56640 1 -- 172.21.15.5:0/1488523483 --> [v2:172.21.15.6:6800/3179572934,v1:172.21.15.6:6801/3179572934] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f094415ed30 con 0x7f0924073060 2023-11-24T06:15:59.031 INFO:teuthology.orchestra.run.smithi005.stdout:60129542149 2023-11-24T06:15:59.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T06:15:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.006+0000 7f0932ffd640 1 -- 172.21.15.5:0/1488523483 <== osd.5 v2:172.21.15.6:6800/3179572934 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f094415ed30 con 0x7f0924073060 2023-11-24T06:15:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.007+0000 7f0948e56640 1 -- 172.21.15.5:0/1488523483 >> [v2:172.21.15.6:6800/3179572934,v1:172.21.15.6:6801/3179572934] conn(0x7f0924073060 msgr2=0x7f0924075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.007+0000 7f0948e56640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.6:6800/3179572934,v1:172.21.15.6:6801/3179572934] conn(0x7f0924073060 0x7f0924075480 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.007+0000 7f0948e56640 1 -- 172.21.15.5:0/1488523483 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f092406ae40 msgr2=0x7f092406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.007+0000 7f0948e56640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f092406ae40 0x7f092406d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0944068d90 tx=0x7f0934008040 comp rx=0 tx=0).stop 2023-11-24T06:15:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.007+0000 7f0948e56640 1 -- 172.21.15.5:0/1488523483 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09441f5fd0 msgr2=0x7f09441f6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.007+0000 7f0948e56640 1 --2- 172.21.15.5:0/1488523483 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09441f5fd0 0x7f09441f6430 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0938001d80 tx=0x7f0938001930 comp rx=0 tx=0).stop 2023-11-24T06:15:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.007+0000 7f0942d9d640 1 -- 172.21.15.5:0/1488523483 reap_dead start 2023-11-24T06:15:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.007+0000 7f0948e56640 1 -- 172.21.15.5:0/1488523483 shutdown_connections 2023-11-24T06:15:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.007+0000 7f0948e56640 1 -- 172.21.15.5:0/1488523483 >> 172.21.15.5:0/1488523483 conn(0x7f09440721a0 msgr2=0x7f09440b6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.008+0000 7f0948e56640 1 -- 172.21.15.5:0/1488523483 shutdown_connections 2023-11-24T06:15:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.008+0000 7f0948e56640 1 -- 172.21.15.5:0/1488523483 wait complete. 2023-11-24T06:15:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.008+0000 7fc451aa6640 1 -- 172.21.15.5:0/3289771613 --> [v2:172.21.15.6:6808/4073132861,v1:172.21.15.6:6809/4073132861] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc44c15c7d0 con 0x7fc42c0730b0 2023-11-24T06:15:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.011+0000 7fc448ff9640 1 -- 172.21.15.5:0/3289771613 <== osd.7 v2:172.21.15.6:6808/4073132861 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc44c15c7d0 con 0x7fc42c0730b0 2023-11-24T06:15:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.012+0000 7fc451aa6640 1 -- 172.21.15.5:0/3289771613 >> [v2:172.21.15.6:6808/4073132861,v1:172.21.15.6:6809/4073132861] conn(0x7fc42c0730b0 msgr2=0x7fc42c0754d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.012+0000 7fc451aa6640 1 --2- 172.21.15.5:0/3289771613 >> [v2:172.21.15.6:6808/4073132861,v1:172.21.15.6:6809/4073132861] conn(0x7fc42c0730b0 0x7fc42c0754d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.012+0000 7fc451aa6640 1 -- 172.21.15.5:0/3289771613 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc42c06ae90 msgr2=0x7fc42c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.012+0000 7fc451aa6640 1 --2- 172.21.15.5:0/3289771613 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc42c06ae90 0x7fc42c06d350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc43c006e80 tx=0x7fc43c006dd0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.012+0000 7fc451aa6640 1 -- 172.21.15.5:0/3289771613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc44c0d37b0 msgr2=0x7fc44c1ff930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.012+0000 7fc451aa6640 1 --2- 172.21.15.5:0/3289771613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc44c0d37b0 0x7fc44c1ff930 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc43400b410 tx=0x7fc43400b8d0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.012+0000 7fc44bfff640 1 -- 172.21.15.5:0/3289771613 reap_dead start 2023-11-24T06:15:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.012+0000 7fc451aa6640 1 -- 172.21.15.5:0/3289771613 shutdown_connections 2023-11-24T06:15:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.012+0000 7fc451aa6640 1 -- 172.21.15.5:0/3289771613 >> 172.21.15.5:0/3289771613 conn(0x7fc44c0721a0 msgr2=0x7fc44c0cfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.013+0000 7fc451aa6640 1 -- 172.21.15.5:0/3289771613 shutdown_connections 2023-11-24T06:15:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.013+0000 7fc451aa6640 1 -- 172.21.15.5:0/3289771613 wait complete. 2023-11-24T06:15:59.035 INFO:teuthology.orchestra.run.smithi005.stdout:60129542150 2023-11-24T06:15:59.035 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T06:15:59.036 INFO:teuthology.orchestra.run.smithi005.stdout:60129542149 2023-11-24T06:15:59.036 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T06:15:59.037 INFO:teuthology.orchestra.run.smithi005.stdout:64424509446 2023-11-24T06:15:59.037 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T06:15:59.038 INFO:teuthology.orchestra.run.smithi005.stdout:60129542149 2023-11-24T06:15:59.038 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T06:15:59.039 INFO:teuthology.orchestra.run.smithi005.stdout:60129542151 2023-11-24T06:15:59.039 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T06:15:59.040 INFO:teuthology.orchestra.run.smithi005.stdout:64424509445 2023-11-24T06:15:59.040 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T06:15:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.375+0000 7f1c113d5640 1 Processor -- start 2023-11-24T06:15:59.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.376+0000 7f1c113d5640 1 -- start start 2023-11-24T06:15:59.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.377+0000 7fe9974e0640 1 Processor -- start 2023-11-24T06:15:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.378+0000 7f1c113d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c0c159240 0x7f1c0c15cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.377+0000 7fe9974e0640 1 -- start start 2023-11-24T06:15:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.379+0000 7f1c113d5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1c0c0ce6a0 0x7f1c0c15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.379+0000 7f1c113d5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c0c0b3160 con 0x7f1c0c15d820 2023-11-24T06:15:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.379+0000 7fe9974e0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9880df100 0x7fe9880e0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.380+0000 7f1c113d5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c0c0cb6d0 con 0x7f1c0c0ce6a0 2023-11-24T06:15:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.380+0000 7f1c113d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c0b3440 con 0x7f1c0c159240 2023-11-24T06:15:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.380+0000 7fe9974e0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe988050180 0x7fe988005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.380+0000 7f1c0b7fe640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f1c0c15d820 0x7f1c0c070110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.5:45720/0 (socket says 172.21.15.5:45720) 2023-11-24T06:15:59.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.380+0000 7f1c0b7fe640 1 -- 172.21.15.5:0/3022606779 learned_addr learned my addr 172.21.15.5:0/3022606779 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.380+0000 7f1c0a7fc640 1 --2- 172.21.15.5:0/3022606779 >> v2:172.21.15.6:3300/0 conn(0x7f1c0c0ce6a0 0x7f1c0c15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.380+0000 7fe9974e0640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9880df4e0 con 0x7fe9880530a0 2023-11-24T06:15:59.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.380+0000 7f1c0affd640 1 --2- 172.21.15.5:0/3022606779 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c0c159240 0x7f1c0c15cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.381+0000 7fe9974e0640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98804f840 con 0x7fe988050180 2023-11-24T06:15:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.381+0000 7fe99527c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9880df100 0x7fe9880e0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.381+0000 7fe9974e0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880e0640 con 0x7fe9880df100 2023-11-24T06:15:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.381+0000 7fe994a7b640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fe9880530a0 0x7fe98804fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.5:45728/0 (socket says 172.21.15.5:45728) 2023-11-24T06:15:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.381+0000 7fe995a7d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe988050180 0x7fe988005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.381+0000 7fe995a7d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe988050180 0x7fe988005fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35806/0 (socket says 172.21.15.5:35806) 2023-11-24T06:15:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.381+0000 7fe995a7d640 1 -- 172.21.15.5:0/1976532023 learned_addr learned my addr 172.21.15.5:0/1976532023 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.382+0000 7fe995a7d640 1 -- 172.21.15.5:0/1976532023 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9880df100 msgr2=0x7fe9880e0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:59.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.382+0000 7fe995a7d640 1 --2- 172.21.15.5:0/1976532023 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9880df100 0x7fe9880e0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.382+0000 7fe995a7d640 1 -- 172.21.15.5:0/1976532023 >> v1:172.21.15.6:6789/0 conn(0x7fe9880530a0 legacy=0x7fe98804fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.383+0000 7fe995a7d640 1 -- 172.21.15.5:0/1976532023 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9880535f0 con 0x7fe988050180 2023-11-24T06:15:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.381+0000 7f1c0a7fc640 1 -- 172.21.15.5:0/3022606779 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c0c159240 msgr2=0x7f1c0c15cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.382+0000 7f1c0a7fc640 1 --2- 172.21.15.5:0/3022606779 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c0c159240 0x7f1c0c15cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.382+0000 7f1c0a7fc640 1 -- 172.21.15.5:0/3022606779 >> v1:172.21.15.6:6789/0 conn(0x7f1c0c15d820 legacy=0x7f1c0c070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.382+0000 7f1c0a7fc640 1 -- 172.21.15.5:0/3022606779 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0c0b3720 con 0x7f1c0c0ce6a0 2023-11-24T06:15:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.383+0000 7f1c0affd640 1 --2- 172.21.15.5:0/3022606779 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c0c159240 0x7f1c0c15cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:15:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.383+0000 7f1c0a7fc640 1 --2- 172.21.15.5:0/3022606779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c0c0ce6a0 0x7f1c0c15d2e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc007040 tx=0x7f1bfc02b6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e30b5e7e72b1027 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.384+0000 7fe995a7d640 1 --2- 172.21.15.5:0/1976532023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe988050180 0x7fe988005fe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe9900691e0 tx=0x7fe990069940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76234c1d3628da74 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.384+0000 7fe987fff640 1 -- 172.21.15.5:0/1976532023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9900b0640 con 0x7fe988050180 2023-11-24T06:15:59.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.384+0000 7fe987fff640 1 -- 172.21.15.5:0/1976532023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9900b07a0 con 0x7fe988050180 2023-11-24T06:15:59.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.384+0000 7fe987fff640 1 -- 172.21.15.5:0/1976532023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9900b0bf0 con 0x7fe988050180 2023-11-24T06:15:59.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.385+0000 7fe9974e0640 1 -- 172.21.15.5:0/1976532023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe988050180 msgr2=0x7fe988005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.385+0000 7fe9974e0640 1 --2- 172.21.15.5:0/1976532023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe988050180 0x7fe988005fe0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe9900691e0 tx=0x7fe990069940 comp rx=0 tx=0).stop 2023-11-24T06:15:59.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.386+0000 7fe9974e0640 1 -- 172.21.15.5:0/1976532023 shutdown_connections 2023-11-24T06:15:59.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.386+0000 7fe9974e0640 1 --2- 172.21.15.5:0/1976532023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe988050180 0x7fe988005fe0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.386+0000 7fe9974e0640 1 --2- 172.21.15.5:0/1976532023 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9880df100 0x7fe9880e0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.386+0000 7fe9974e0640 1 -- 172.21.15.5:0/1976532023 >> 172.21.15.5:0/1976532023 conn(0x7fe988009fb0 msgr2=0x7fe98800a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.386+0000 7f1c09ffb640 1 -- 172.21.15.5:0/3022606779 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc007ce0 con 0x7f1c0c0ce6a0 2023-11-24T06:15:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.386+0000 7fe9974e0640 1 -- 172.21.15.5:0/1976532023 shutdown_connections 2023-11-24T06:15:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.386+0000 7fe9974e0640 1 -- 172.21.15.5:0/1976532023 wait complete. 2023-11-24T06:15:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.387+0000 7f1c09ffb640 1 -- 172.21.15.5:0/3022606779 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bfc02e050 con 0x7f1c0c0ce6a0 2023-11-24T06:15:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.387+0000 7f1c09ffb640 1 -- 172.21.15.5:0/3022606779 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0092f0 con 0x7f1c0c0ce6a0 2023-11-24T06:15:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.388+0000 7f1c113d5640 1 -- 172.21.15.5:0/3022606779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c0c0ce6a0 msgr2=0x7f1c0c15d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.388+0000 7fe9974e0640 1 Processor -- start 2023-11-24T06:15:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.388+0000 7fe9974e0640 1 -- start start 2023-11-24T06:15:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.388+0000 7f1c113d5640 1 --2- 172.21.15.5:0/3022606779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c0c0ce6a0 0x7f1c0c15d2e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc007040 tx=0x7f1bfc02b6d0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.388+0000 7fbe725b0640 1 Processor -- start 2023-11-24T06:15:59.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.389+0000 7fe9974e0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe988050180 0x7fe98817ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.389+0000 7fe9974e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9880530a0 0x7fe98804ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.389+0000 7fe9974e0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9880df100 0x7fe98804b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.389+0000 7fe9974e0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880c77b0 con 0x7fe9880530a0 2023-11-24T06:15:59.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.389+0000 7fe9974e0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98804d570 con 0x7fe988050180 2023-11-24T06:15:59.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.389+0000 7fe9974e0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98804d290 con 0x7fe9880df100 2023-11-24T06:15:59.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.390+0000 7fe994a7b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9880530a0 0x7fe98804ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.390+0000 7fe994a7b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9880530a0 0x7fe98804ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35816/0 (socket says 172.21.15.5:35816) 2023-11-24T06:15:59.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.390+0000 7fe994a7b640 1 -- 172.21.15.5:0/3342569425 learned_addr learned my addr 172.21.15.5:0/3342569425 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.388+0000 7fbe725b0640 1 -- start start 2023-11-24T06:15:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.391+0000 7f1c113d5640 1 -- 172.21.15.5:0/3022606779 shutdown_connections 2023-11-24T06:15:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.391+0000 7f1c113d5640 1 --2- 172.21.15.5:0/3022606779 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c0c0ce6a0 0x7f1c0c15d2e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.391+0000 7f1c113d5640 1 --2- 172.21.15.5:0/3022606779 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c0c159240 0x7f1c0c15cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.391+0000 7f1c113d5640 1 -- 172.21.15.5:0/3022606779 >> 172.21.15.5:0/3022606779 conn(0x7f1c0c0b0840 msgr2=0x7f1c0c0b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.390+0000 7fe994a7b640 1 -- 172.21.15.5:0/3342569425 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9880df100 msgr2=0x7fe98804b060 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:15:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.390+0000 7fe994a7b640 1 --2- 172.21.15.5:0/3342569425 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9880df100 0x7fe98804b060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.390+0000 7fe994a7b640 1 -- 172.21.15.5:0/3342569425 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe988050180 msgr2=0x7fe98817ce40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:15:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.390+0000 7fe994a7b640 1 --2- 172.21.15.5:0/3342569425 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe988050180 0x7fe98817ce40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.390+0000 7fe994a7b640 1 -- 172.21.15.5:0/3342569425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99004f730 con 0x7fe9880530a0 2023-11-24T06:15:59.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.391+0000 7fe994a7b640 1 --2- 172.21.15.5:0/3342569425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9880530a0 0x7fe98804ab20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe98000c450 tx=0x7fe98000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.391+0000 7f1c113d5640 1 -- 172.21.15.5:0/3022606779 shutdown_connections 2023-11-24T06:15:59.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.391+0000 7f1c113d5640 1 -- 172.21.15.5:0/3022606779 wait complete. 2023-11-24T06:15:59.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.391+0000 7fe985ffb640 1 -- 172.21.15.5:0/3342569425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9800074e0 con 0x7fe9880530a0 2023-11-24T06:15:59.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.392+0000 7fe985ffb640 1 -- 172.21.15.5:0/3342569425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe980007640 con 0x7fe9880530a0 2023-11-24T06:15:59.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.392+0000 7fe985ffb640 1 -- 172.21.15.5:0/3342569425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980005a50 con 0x7fe9880530a0 2023-11-24T06:15:59.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.392+0000 7fbe725b0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe640df0c0 0x7fbe640e00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.392+0000 7fbe725b0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe640df4a0 con 0x7fbe64053060 2023-11-24T06:15:59.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.392+0000 7fbe725b0640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe640535b0 con 0x7fbe64050140 2023-11-24T06:15:59.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.392+0000 7fbe725b0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6404f800 con 0x7fbe640df0c0 2023-11-24T06:15:59.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.392+0000 7fbe70b4d640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fbe64050140 0x7fbe64005fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.5:45740/0 (socket says 172.21.15.5:45740) 2023-11-24T06:15:59.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.393+0000 7fbe70b4d640 1 -- 172.21.15.5:0/189134957 learned_addr learned my addr 172.21.15.5:0/189134957 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:59.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.393+0000 7fbe6bfff640 1 --2- 172.21.15.5:0/189134957 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe640df0c0 0x7fbe640e00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.392+0000 7fe9974e0640 1 -- 172.21.15.5:0/3342569425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9880c70f0 con 0x7fe9880530a0 2023-11-24T06:15:59.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.392+0000 7fe9974e0640 1 -- 172.21.15.5:0/3342569425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98817d380 con 0x7fe9880530a0 2023-11-24T06:15:59.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.394+0000 7f1c113d5640 1 Processor -- start 2023-11-24T06:15:59.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.394+0000 7fbe6bfff640 1 -- 172.21.15.5:0/189134957 >> v1:172.21.15.6:6789/0 conn(0x7fbe64050140 legacy=0x7fbe64005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.395+0000 7fe9974e0640 1 -- 172.21.15.5:0/3342569425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9880c78f0 con 0x7fe9880530a0 2023-11-24T06:15:59.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.395+0000 7f1c113d5640 1 -- start start 2023-11-24T06:15:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.394+0000 7fbe6affd640 1 -- 172.21.15.5:0/189134957 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647718477 0 0) 0x7fbe640535b0 con 0x7fbe64050140 2023-11-24T06:15:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.394+0000 7fbe6bfff640 1 -- 172.21.15.5:0/189134957 >> v1:172.21.15.5:6789/0 conn(0x7fbe64053060 legacy=0x7fbe6404fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.394+0000 7fbe6bfff640 1 -- 172.21.15.5:0/189134957 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe64053890 con 0x7fbe640df0c0 2023-11-24T06:15:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.396+0000 7fbe6bfff640 1 --2- 172.21.15.5:0/189134957 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe640df0c0 0x7fbe640e00c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000c470 tx=0x7fbe6002cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a642786892d6ff70 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.397+0000 7fbe6affd640 1 -- 172.21.15.5:0/189134957 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6002d9f0 con 0x7fbe640df0c0 2023-11-24T06:15:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.397+0000 7fbe6affd640 1 -- 172.21.15.5:0/189134957 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6002db50 con 0x7fbe640df0c0 2023-11-24T06:15:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.397+0000 7fbe6affd640 1 -- 172.21.15.5:0/189134957 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe60035440 con 0x7fbe640df0c0 2023-11-24T06:15:59.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.398+0000 7fbe725b0640 1 -- 172.21.15.5:0/189134957 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe640df0c0 msgr2=0x7fbe640e00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.398+0000 7fbe725b0640 1 --2- 172.21.15.5:0/189134957 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe640df0c0 0x7fbe640e00c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000c470 tx=0x7fbe6002cc40 comp rx=0 tx=0).stop 2023-11-24T06:15:59.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.399+0000 7fbe725b0640 1 -- 172.21.15.5:0/189134957 shutdown_connections 2023-11-24T06:15:59.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.399+0000 7fbe725b0640 1 --2- 172.21.15.5:0/189134957 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe640df0c0 0x7fbe640e00c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.399+0000 7fbe725b0640 1 -- 172.21.15.5:0/189134957 >> 172.21.15.5:0/189134957 conn(0x7fbe64009f70 msgr2=0x7fbe6400a380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.396+0000 7f1c113d5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c0c0ce6a0 0x7f1c0c14fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.399+0000 7fbe725b0640 1 -- 172.21.15.5:0/189134957 shutdown_connections 2023-11-24T06:15:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.396+0000 7f1c113d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c159240 0x7f1c0c14e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.396+0000 7f1c113d5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c0c15d820 0x7f1c0c1514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.397+0000 7f1c113d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c1502c0 con 0x7f1c0c159240 2023-11-24T06:15:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.397+0000 7f1c113d5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c14d5e0 con 0x7f1c0c0ce6a0 2023-11-24T06:15:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.397+0000 7f1c113d5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c15bbc0 con 0x7f1c0c15d820 2023-11-24T06:15:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.397+0000 7f1c0b7fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c0c15d820 0x7f1c0c1514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.397+0000 7f1c0b7fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c0c15d820 0x7f1c0c1514b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:53428/0 (socket says 172.21.15.5:53428) 2023-11-24T06:15:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.397+0000 7f1c0b7fe640 1 -- 172.21.15.5:0/1773712705 learned_addr learned my addr 172.21.15.5:0/1773712705 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.398+0000 7f1c0b7fe640 1 -- 172.21.15.5:0/1773712705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c0c0ce6a0 msgr2=0x7f1c0c14fd80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:15:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.398+0000 7f1c0b7fe640 1 --2- 172.21.15.5:0/1773712705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c0c0ce6a0 0x7f1c0c14fd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.398+0000 7f1c0b7fe640 1 -- 172.21.15.5:0/1773712705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c159240 msgr2=0x7f1c0c14e150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T06:15:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.398+0000 7f1c0b7fe640 1 --2- 172.21.15.5:0/1773712705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c159240 0x7f1c0c14e150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.398+0000 7f1c0b7fe640 1 -- 172.21.15.5:0/1773712705 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc009d20 con 0x7f1c0c15d820 2023-11-24T06:15:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.399+0000 7fbe725b0640 1 -- 172.21.15.5:0/189134957 wait complete. 2023-11-24T06:15:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.400+0000 7fe985ffb640 1 -- 172.21.15.5:0/3342569425 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe980005bb0 con 0x7fe9880530a0 2023-11-24T06:15:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.400+0000 7f1c0b7fe640 1 --2- 172.21.15.5:0/1773712705 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c0c15d820 0x7f1c0c1514b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000c5a0 tx=0x7f1c0000ca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.401+0000 7f1bf37fe640 1 -- 172.21.15.5:0/1773712705 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c000072e0 con 0x7f1c0c15d820 2023-11-24T06:15:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.401+0000 7f1c113d5640 1 -- 172.21.15.5:0/1773712705 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0c14d300 con 0x7f1c0c15d820 2023-11-24T06:15:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.401+0000 7f1c113d5640 1 -- 172.21.15.5:0/1773712705 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0c159c20 con 0x7f1c0c15d820 2023-11-24T06:15:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.401+0000 7f1bf37fe640 1 -- 172.21.15.5:0/1773712705 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c00007440 con 0x7f1c0c15d820 2023-11-24T06:15:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.401+0000 7f1bf37fe640 1 -- 172.21.15.5:0/1773712705 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c000072e0 con 0x7f1c0c15d820 2023-11-24T06:15:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.403+0000 7f4e22333640 1 Processor -- start 2023-11-24T06:15:59.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.402+0000 7fbe725b0640 1 Processor -- start 2023-11-24T06:15:59.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.403+0000 7f4e22333640 1 -- start start 2023-11-24T06:15:59.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.404+0000 7fbe725b0640 1 -- start start 2023-11-24T06:15:59.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.404+0000 7f1c113d5640 1 -- 172.21.15.5:0/1773712705 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bec005250 con 0x7f1c0c15d820 2023-11-24T06:15:59.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.403+0000 7fe985ffb640 1 --2- 172.21.15.5:0/3342569425 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe97406af10 0x7fe97406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.404+0000 7fe985ffb640 1 -- 172.21.15.5:0/3342569425 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe980088f10 con 0x7fe9880530a0 2023-11-24T06:15:59.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.405+0000 7f4e22333640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e1c15d4d0 0x7f4e1c0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.405+0000 7f4e22333640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4e1c0ce260 0x7f4e1c0d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.405+0000 7f4e22333640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e1c15d8b0 con 0x7f4e1c0ce6c0 2023-11-24T06:15:59.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.405+0000 7fbe725b0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe64050140 0x7fbe64178b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.405+0000 7fbe725b0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe64053060 0x7fbe640cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.405+0000 7fbe725b0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe640df0c0 0x7fbe6404ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.405+0000 7fbe725b0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6404a860 con 0x7fbe64053060 2023-11-24T06:15:59.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.406+0000 7fbe725b0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe640ccf90 con 0x7fbe640df0c0 2023-11-24T06:15:59.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.406+0000 7fbe725b0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe64049be0 con 0x7fbe64050140 2023-11-24T06:15:59.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.405+0000 7f4e22333640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e1c0b91b0 con 0x7f4e1c0ce260 2023-11-24T06:15:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.405+0000 7f4e22333640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c0cdfb0 con 0x7f4e1c15d4d0 2023-11-24T06:15:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.406+0000 7f4e1bfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e1c15d4d0 0x7f4e1c0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.406+0000 7f4e1bfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e1c15d4d0 0x7f4e1c0cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:53436/0 (socket says 172.21.15.5:53436) 2023-11-24T06:15:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.406+0000 7f4e1bfff640 1 -- 172.21.15.5:0/495341589 learned_addr learned my addr 172.21.15.5:0/495341589 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.406+0000 7f4e137fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4e1c0ce260 0x7f4e1c0d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.406+0000 7fbe70b4d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe640df0c0 0x7fbe6404ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.406+0000 7fbe70b4d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe640df0c0 0x7fbe6404ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:50092/0 (socket says 172.21.15.5:50092) 2023-11-24T06:15:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.407+0000 7fbe6b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe64053060 0x7fbe640cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.405+0000 7fe99527c640 1 --2- 172.21.15.5:0/3342569425 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe97406af10 0x7fe97406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.407+0000 7fbe6b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe64053060 0x7fbe640cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35840/0 (socket says 172.21.15.5:35840) 2023-11-24T06:15:59.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.406+0000 7f4e137fe640 1 -- 172.21.15.5:0/495341589 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e1c15d4d0 msgr2=0x7f4e1c0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:59.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.406+0000 7f4e137fe640 1 --2- 172.21.15.5:0/495341589 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e1c15d4d0 0x7f4e1c0cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.406+0000 7f4e137fe640 1 -- 172.21.15.5:0/495341589 >> v1:172.21.15.6:6789/0 conn(0x7f4e1c0ce6c0 legacy=0x7f4e1c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:59.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.407+0000 7f4e137fe640 1 -- 172.21.15.5:0/495341589 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1c0b2400 con 0x7f4e1c0ce260 2023-11-24T06:15:59.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.407+0000 7fbe6b7fe640 1 -- 172.21.15.5:0/787532581 learned_addr learned my addr 172.21.15.5:0/787532581 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:59.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.407+0000 7fbe6bfff640 1 --2- 172.21.15.5:0/787532581 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe64050140 0x7fbe64178b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.407+0000 7fbe6b7fe640 1 -- 172.21.15.5:0/787532581 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe64050140 msgr2=0x7fbe64178b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.407+0000 7fbe6b7fe640 1 --2- 172.21.15.5:0/787532581 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe64050140 0x7fbe64178b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.408+0000 7fbe6b7fe640 1 -- 172.21.15.5:0/787532581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe640df0c0 msgr2=0x7fbe6404ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.408+0000 7fbe6b7fe640 1 --2- 172.21.15.5:0/787532581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe640df0c0 0x7fbe6404ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.408+0000 7fbe6b7fe640 1 -- 172.21.15.5:0/787532581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe60007cc0 con 0x7fbe64053060 2023-11-24T06:15:59.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.406+0000 7f1bf37fe640 1 -- 172.21.15.5:0/1773712705 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c000047e0 con 0x7f1c0c15d820 2023-11-24T06:15:59.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.408+0000 7fbe6b7fe640 1 --2- 172.21.15.5:0/787532581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe64053060 0x7fbe640cc6b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c400 tx=0x7fbe5800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.408+0000 7f1bf37fe640 1 --2- 172.21.15.5:0/1773712705 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1bf806aea0 0x7f1bf806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.408+0000 7f4e137fe640 1 --2- 172.21.15.5:0/495341589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c0ce260 0x7f4e1c0d1850 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0097a0 tx=0x7f4e0c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e3b1ae85c160c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.409+0000 7fe985ffb640 1 -- 172.21.15.5:0/3342569425 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe980054030 con 0x7fe9880530a0 2023-11-24T06:15:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.409+0000 7f8256cc6640 1 Processor -- start 2023-11-24T06:15:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.409+0000 7fe99527c640 1 --2- 172.21.15.5:0/3342569425 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe97406af10 0x7fe97406d3d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009870 tx=0x7fe98c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.409+0000 7f8256cc6640 1 -- start start 2023-11-24T06:15:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.409+0000 7fbe68ff9640 1 -- 172.21.15.5:0/787532581 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe580052d0 con 0x7fbe64053060 2023-11-24T06:15:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.409+0000 7f1c0affd640 1 --2- 172.21.15.5:0/1773712705 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1bf806aea0 0x7f1bf806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.409+0000 7fbe68ff9640 1 -- 172.21.15.5:0/787532581 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe58005450 con 0x7fbe64053060 2023-11-24T06:15:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.409+0000 7fbe68ff9640 1 -- 172.21.15.5:0/787532581 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe58003830 con 0x7fbe64053060 2023-11-24T06:15:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.409+0000 7fbe725b0640 1 -- 172.21.15.5:0/787532581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe64049ec0 con 0x7fbe64053060 2023-11-24T06:15:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.409+0000 7fbe725b0640 1 -- 172.21.15.5:0/787532581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6404a480 con 0x7fbe64053060 2023-11-24T06:15:59.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.410+0000 7f1bf37fe640 1 -- 172.21.15.5:0/1773712705 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c00068420 con 0x7f1c0c15d820 2023-11-24T06:15:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.410+0000 7f4e1b7fe640 1 -- 172.21.15.5:0/495341589 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c031780 con 0x7f4e1c0ce260 2023-11-24T06:15:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.410+0000 7f4e1b7fe640 1 -- 172.21.15.5:0/495341589 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c0318e0 con 0x7f4e1c0ce260 2023-11-24T06:15:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.410+0000 7f4e1b7fe640 1 -- 172.21.15.5:0/495341589 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c031780 con 0x7f4e1c0ce260 2023-11-24T06:15:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.411+0000 7fbe725b0640 1 -- 172.21.15.5:0/787532581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6404a9a0 con 0x7fbe64053060 2023-11-24T06:15:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.412+0000 7f1c0affd640 1 --2- 172.21.15.5:0/1773712705 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1bf806aea0 0x7f1bf806d360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0b2090 tx=0x7f1c04007300 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.411+0000 7f4e22333640 1 -- 172.21.15.5:0/495341589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c0ce260 msgr2=0x7f4e1c0d1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.411+0000 7f4e22333640 1 --2- 172.21.15.5:0/495341589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c0ce260 0x7f4e1c0d1850 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0097a0 tx=0x7f4e0c02b820 comp rx=0 tx=0).stop 2023-11-24T06:15:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.411+0000 7f8256cc6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f825015d4d0 0x7f82500cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.412+0000 7f8256cc6640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f82500d1f60 0x7f82500b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.412+0000 7f8256cc6640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825015d8b0 con 0x7f82500ce260 2023-11-24T06:15:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.412+0000 7f8256cc6640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82500b91b0 con 0x7f825015d4d0 2023-11-24T06:15:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.412+0000 7f8256cc6640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82500cdfb0 con 0x7f82500d1f60 2023-11-24T06:15:59.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.411+0000 7f7b413db640 1 Processor -- start 2023-11-24T06:15:59.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.411+0000 7f7b413db640 1 -- start start 2023-11-24T06:15:59.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.412+0000 7f4e22333640 1 -- 172.21.15.5:0/495341589 shutdown_connections 2023-11-24T06:15:59.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.412+0000 7f4e22333640 1 --2- 172.21.15.5:0/495341589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c0ce260 0x7f4e1c0d1850 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.412+0000 7f4e22333640 1 --2- 172.21.15.5:0/495341589 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e1c15d4d0 0x7f4e1c0cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.412+0000 7f4e22333640 1 -- 172.21.15.5:0/495341589 >> 172.21.15.5:0/495341589 conn(0x7f4e1c071f90 msgr2=0x7f4e1c0723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.413+0000 7f4e22333640 1 -- 172.21.15.5:0/495341589 shutdown_connections 2023-11-24T06:15:59.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.413+0000 7f4e22333640 1 -- 172.21.15.5:0/495341589 wait complete. 2023-11-24T06:15:59.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.413+0000 7f8255263640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f82500d1f60 0x7f82500b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.413+0000 7f8255263640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f82500d1f60 0x7f82500b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:50098/0 (socket says 172.21.15.5:50098) 2023-11-24T06:15:59.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.413+0000 7f8255263640 1 -- 172.21.15.5:0/1367078603 learned_addr learned my addr 172.21.15.5:0/1367078603 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:59.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.414+0000 7f8255263640 1 -- 172.21.15.5:0/1367078603 >> v2:172.21.15.5:3300/0 conn(0x7f825015d4d0 msgr2=0x7f82500cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:59.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.414+0000 7f8255263640 1 --2- 172.21.15.5:0/1367078603 >> v2:172.21.15.5:3300/0 conn(0x7f825015d4d0 0x7f82500cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.414+0000 7f7b413db640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7b34006670 0x7f7b3403c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.414+0000 7f7b413db640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b34007870 con 0x7f7b340061e0 2023-11-24T06:15:59.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.414+0000 7f7b413db640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b34006c60 con 0x7f7b34053780 2023-11-24T06:15:59.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.414+0000 7f7b413db640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b34006f40 con 0x7f7b34006670 2023-11-24T06:15:59.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.415+0000 7f7b3b7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7b340061e0 0x7f7b3403c0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56562/0 (socket says 172.21.15.5:56562) 2023-11-24T06:15:59.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.415+0000 7f7b3b7fe640 1 -- 172.21.15.5:0/3233662515 learned_addr learned my addr 172.21.15.5:0/3233662515 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:59.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.414+0000 7fbe68ff9640 1 -- 172.21.15.5:0/787532581 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbe58003990 con 0x7fbe64053060 2023-11-24T06:15:59.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.416+0000 7fbe68ff9640 1 --2- 172.21.15.5:0/787532581 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbe5006ae40 0x7fbe5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.414+0000 7f8255263640 1 -- 172.21.15.5:0/1367078603 >> v1:172.21.15.5:6789/0 conn(0x7f82500ce260 legacy=0x7f82500d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:59.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.414+0000 7f8255263640 1 -- 172.21.15.5:0/1367078603 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82500b2400 con 0x7f82500d1f60 2023-11-24T06:15:59.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.416+0000 7f7b40bda640 1 --2- 172.21.15.5:0/3233662515 >> v2:172.21.15.6:3300/0 conn(0x7f7b34006670 0x7f7b3403c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.416+0000 7f4e22333640 1 Processor -- start 2023-11-24T06:15:59.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.416+0000 7f7b3affd640 1 -- 172.21.15.5:0/3233662515 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852241040 0 0) 0x7f7b34006c60 con 0x7f7b34053780 2023-11-24T06:15:59.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.417+0000 7f4e22333640 1 -- start start 2023-11-24T06:15:59.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.417+0000 7f7b3affd640 1 -- 172.21.15.5:0/3233662515 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b2c003520 con 0x7f7b34053780 2023-11-24T06:15:59.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.417+0000 7f8255263640 1 --2- 172.21.15.5:0/1367078603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82500d1f60 0x7f82500b4cf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f825006c470 tx=0x7f824802bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc97159653bb0fa4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.417+0000 7f7b40bda640 1 -- 172.21.15.5:0/3233662515 >> v1:172.21.15.6:6789/0 conn(0x7f7b34053780 legacy=0x7f7b34053b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.417+0000 7f7b40bda640 1 -- 172.21.15.5:0/3233662515 >> v1:172.21.15.5:6789/0 conn(0x7f7b340061e0 legacy=0x7f7b3403c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.416+0000 7fbe6bfff640 1 --2- 172.21.15.5:0/787532581 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbe5006ae40 0x7fbe5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.417+0000 7f7b40bda640 1 -- 172.21.15.5:0/3233662515 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b34007220 con 0x7f7b34006670 2023-11-24T06:15:59.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.417+0000 7fbe68ff9640 1 -- 172.21.15.5:0/787532581 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe58088ea0 con 0x7fbe64053060 2023-11-24T06:15:59.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.418+0000 7f824f7fe640 1 -- 172.21.15.5:0/1367078603 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824802caa0 con 0x7f82500d1f60 2023-11-24T06:15:59.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.418+0000 7f40c17e3640 1 Processor -- start 2023-11-24T06:15:59.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.418+0000 7f824f7fe640 1 -- 172.21.15.5:0/1367078603 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824802cc00 con 0x7f82500d1f60 2023-11-24T06:15:59.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.418+0000 7f824f7fe640 1 -- 172.21.15.5:0/1367078603 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82480343f0 con 0x7f82500d1f60 2023-11-24T06:15:59.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.419+0000 7f4e22333640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e1c0ce260 0x7f4e1c0c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.419+0000 7f4e22333640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e1c0ce6c0 0x7f4e1c0b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.419+0000 7f4e22333640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c15d4d0 0x7f4e1c1f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.419+0000 7f7b40bda640 1 --2- 172.21.15.5:0/3233662515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b34006670 0x7f7b3403c7d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c069dd0 tx=0x7f7b3c0a5e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abb1e25628402c2a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.418+0000 7f40c17e3640 1 -- start start 2023-11-24T06:15:59.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.419+0000 7f8256cc6640 1 -- 172.21.15.5:0/1367078603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82500d1f60 msgr2=0x7f82500b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.419+0000 7f4e208d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c15d4d0 0x7f4e1c1f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.419+0000 7f4e208d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c15d4d0 0x7f4e1c1f2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35862/0 (socket says 172.21.15.5:35862) 2023-11-24T06:15:59.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.419+0000 7f8256cc6640 1 --2- 172.21.15.5:0/1367078603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82500d1f60 0x7f82500b4cf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f825006c470 tx=0x7f824802bf00 comp rx=0 tx=0).stop 2023-11-24T06:15:59.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f4e137fe640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e1c0ce6c0 0x7f4e1c0b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f4e208d0640 1 -- 172.21.15.5:0/3195762394 learned_addr learned my addr 172.21.15.5:0/3195762394 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:59.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f8256cc6640 1 -- 172.21.15.5:0/1367078603 shutdown_connections 2023-11-24T06:15:59.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f8256cc6640 1 --2- 172.21.15.5:0/1367078603 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82500d1f60 0x7f82500b4cf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f8256cc6640 1 --2- 172.21.15.5:0/1367078603 >> v2:172.21.15.5:3300/0 conn(0x7f825015d4d0 0x7f82500cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f8256cc6640 1 -- 172.21.15.5:0/1367078603 >> 172.21.15.5:0/1367078603 conn(0x7f8250071f90 msgr2=0x7f82500723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f4e22333640 1 -- 172.21.15.5:0/3195762394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c0c8040 con 0x7f4e1c15d4d0 2023-11-24T06:15:59.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f4e22333640 1 -- 172.21.15.5:0/3195762394 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c0b7c50 con 0x7f4e1c0ce6c0 2023-11-24T06:15:59.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/3195762394 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e1c0ce260 0x7f4e1c0c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f4e22333640 1 -- 172.21.15.5:0/3195762394 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c0ca7e0 con 0x7f4e1c0ce260 2023-11-24T06:15:59.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f7b3affd640 1 -- 172.21.15.5:0/3233662515 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0a6cf0 con 0x7f7b34006670 2023-11-24T06:15:59.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f40c17e3640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f40bc15d4d0 0x7f40bc0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f40c17e3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f40bc0d1f60 0x7f40bc0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f40c17e3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40bc15d8b0 con 0x7f40bc0ce260 2023-11-24T06:15:59.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f4e208d0640 1 -- 172.21.15.5:0/3195762394 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e1c0ce260 msgr2=0x7f4e1c0c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f4e208d0640 1 --2- 172.21.15.5:0/3195762394 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e1c0ce260 0x7f4e1c0c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f4e208d0640 1 -- 172.21.15.5:0/3195762394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e1c0ce6c0 msgr2=0x7f4e1c0b7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f4e208d0640 1 --2- 172.21.15.5:0/3195762394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e1c0ce6c0 0x7f4e1c0b7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f4e208d0640 1 -- 172.21.15.5:0/3195762394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c009400 con 0x7f4e1c15d4d0 2023-11-24T06:15:59.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.418+0000 7fbe6bfff640 1 --2- 172.21.15.5:0/787532581 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbe5006ae40 0x7fbe5006d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe60037000 tx=0x7fbe60031000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f40c17e3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40bc0b91b0 con 0x7f40bc0d1f60 2023-11-24T06:15:59.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f7b3affd640 1 -- 172.21.15.5:0/3233662515 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3c0a6e50 con 0x7f7b34006670 2023-11-24T06:15:59.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f7b3affd640 1 -- 172.21.15.5:0/3233662515 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c0ae640 con 0x7f7b34006670 2023-11-24T06:15:59.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f7b413db640 1 -- 172.21.15.5:0/3233662515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b34006670 msgr2=0x7f7b3403c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f7b413db640 1 --2- 172.21.15.5:0/3233662515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b34006670 0x7f7b3403c7d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c069dd0 tx=0x7f7b3c0a5e80 comp rx=0 tx=0).stop 2023-11-24T06:15:59.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f7b413db640 1 -- 172.21.15.5:0/3233662515 shutdown_connections 2023-11-24T06:15:59.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f7b413db640 1 --2- 172.21.15.5:0/3233662515 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b34006670 0x7f7b3403c7d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f7b413db640 1 -- 172.21.15.5:0/3233662515 >> 172.21.15.5:0/3233662515 conn(0x7f7b3403aa10 msgr2=0x7f7b3403ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.422+0000 7f7b413db640 1 -- 172.21.15.5:0/3233662515 shutdown_connections 2023-11-24T06:15:59.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.422+0000 7f7b413db640 1 -- 172.21.15.5:0/3233662515 wait complete. 2023-11-24T06:15:59.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f8256cc6640 1 -- 172.21.15.5:0/1367078603 shutdown_connections 2023-11-24T06:15:59.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.420+0000 7f8256cc6640 1 -- 172.21.15.5:0/1367078603 wait complete. 2023-11-24T06:15:59.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.422+0000 7f4e208d0640 1 --2- 172.21.15.5:0/3195762394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c15d4d0 0x7f4e1c1f2a60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400c7b0 tx=0x7f4e1400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f40c17e3640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40bc0cdfb0 con 0x7f40bc15d4d0 2023-11-24T06:15:59.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f40ba7fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f40bc0ce260 0x7f40bc0d1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56576/0 (socket says 172.21.15.5:56576) 2023-11-24T06:15:59.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.421+0000 7f40ba7fc640 1 -- 172.21.15.5:0/199613345 learned_addr learned my addr 172.21.15.5:0/199613345 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:15:59.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.422+0000 7f40baffd640 1 --2- 172.21.15.5:0/199613345 >> v2:172.21.15.6:3300/0 conn(0x7f40bc15d4d0 0x7f40bc0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.423+0000 7f40bb7fe640 1 --2- 172.21.15.5:0/199613345 >> v2:172.21.15.5:3300/0 conn(0x7f40bc0d1f60 0x7f40bc0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.423+0000 7f8256cc6640 1 Processor -- start 2023-11-24T06:15:59.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.423+0000 7f8256cc6640 1 -- start start 2023-11-24T06:15:59.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.423+0000 7f40bb7fe640 1 -- 172.21.15.5:0/199613345 >> v2:172.21.15.6:3300/0 conn(0x7f40bc15d4d0 msgr2=0x7f40bc0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:59.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.423+0000 7f40bb7fe640 1 --2- 172.21.15.5:0/199613345 >> v2:172.21.15.6:3300/0 conn(0x7f40bc15d4d0 0x7f40bc0cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.423+0000 7f40bb7fe640 1 -- 172.21.15.5:0/199613345 >> v1:172.21.15.5:6789/0 conn(0x7f40bc0ce260 legacy=0x7f40bc0d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.423+0000 7f40bb7fe640 1 -- 172.21.15.5:0/199613345 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc0b2400 con 0x7f40bc0d1f60 2023-11-24T06:15:59.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.424+0000 7f7b413db640 1 Processor -- start 2023-11-24T06:15:59.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.424+0000 7f4e197fa640 1 -- 172.21.15.5:0/3195762394 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14003d80 con 0x7f4e1c15d4d0 2023-11-24T06:15:59.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.424+0000 7f4e197fa640 1 -- 172.21.15.5:0/3195762394 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e1400ce60 con 0x7f4e1c15d4d0 2023-11-24T06:15:59.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.424+0000 7f4e197fa640 1 -- 172.21.15.5:0/3195762394 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e140052b0 con 0x7f4e1c15d4d0 2023-11-24T06:15:59.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.424+0000 7f8256cc6640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82500ce260 0x7f82501602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.424+0000 7f8256cc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82500d1f60 0x7f82500ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.425+0000 7fbe68ff9640 1 -- 172.21.15.5:0/787532581 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbe58053fc0 con 0x7fbe64053060 2023-11-24T06:15:59.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.425+0000 7f7b413db640 1 -- start start 2023-11-24T06:15:59.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.425+0000 7f4e22333640 1 -- 172.21.15.5:0/3195762394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1c0c7430 con 0x7f4e1c15d4d0 2023-11-24T06:15:59.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.425+0000 7f4e22333640 1 -- 172.21.15.5:0/3195762394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1c0c79f0 con 0x7f4e1c15d4d0 2023-11-24T06:15:59.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.425+0000 7f8256cc6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f825015d4d0 0x7f8250153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.425+0000 7f8256cc6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825015f050 con 0x7f82500d1f60 2023-11-24T06:15:59.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.425+0000 7f8256cc6640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825015f330 con 0x7f825015d4d0 2023-11-24T06:15:59.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.425+0000 7f8256cc6640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825015f610 con 0x7f82500ce260 2023-11-24T06:15:59.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.426+0000 7f40bb7fe640 1 --2- 172.21.15.5:0/199613345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40bc0d1f60 0x7f40bc0b4cf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f40bc06c470 tx=0x7f40b402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b510b4e06365f150 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.426+0000 7f8254a62640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82500ce260 0x7f82501602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.426+0000 7f8254a62640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82500ce260 0x7f82501602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:53442/0 (socket says 172.21.15.5:53442) 2023-11-24T06:15:59.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.426+0000 7f8254a62640 1 -- 172.21.15.5:0/1567013879 learned_addr learned my addr 172.21.15.5:0/1567013879 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:59.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.426+0000 7f824ffff640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82500d1f60 0x7f82500ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.426+0000 7f8255263640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f825015d4d0 0x7f8250153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.427+0000 7f7b413db640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b340061e0 0x7f7b3404bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.427+0000 7f8254a62640 1 -- 172.21.15.5:0/1567013879 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f825015d4d0 msgr2=0x7f8250153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.427+0000 7f40b9ffb640 1 -- 172.21.15.5:0/199613345 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b402cc70 con 0x7f40bc0d1f60 2023-11-24T06:15:59.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.427+0000 7f40b9ffb640 1 -- 172.21.15.5:0/199613345 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b402cdd0 con 0x7f40bc0d1f60 2023-11-24T06:15:59.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.427+0000 7f40b9ffb640 1 -- 172.21.15.5:0/199613345 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4034660 con 0x7f40bc0d1f60 2023-11-24T06:15:59.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.427+0000 7f8254a62640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f825015d4d0 0x7f8250153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.427+0000 7f4e22333640 1 -- 172.21.15.5:0/3195762394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e1c0c81f0 con 0x7f4e1c15d4d0 2023-11-24T06:15:59.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.427+0000 7f8254a62640 1 -- 172.21.15.5:0/1567013879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82500d1f60 msgr2=0x7f82500ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.427+0000 7f7b413db640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b34006670 0x7f7b3404ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.428+0000 7f7b413db640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b34053780 0x7f7b3404b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.427+0000 7f7f51ce7640 1 Processor -- start 2023-11-24T06:15:59.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.428+0000 7f7f51ce7640 1 -- start start 2023-11-24T06:15:59.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.428+0000 7f7b413db640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3404c270 con 0x7f7b340061e0 2023-11-24T06:15:59.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.428+0000 7f7b413db640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340d9f30 con 0x7f7b34053780 2023-11-24T06:15:59.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.428+0000 7f7b413db640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b340d7770 con 0x7f7b34006670 2023-11-24T06:15:59.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.427+0000 7f8254a62640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82500d1f60 0x7f82500ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.427+0000 7f8254a62640 1 -- 172.21.15.5:0/1567013879 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8248009b30 con 0x7f82500ce260 2023-11-24T06:15:59.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.428+0000 7f824ffff640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82500d1f60 0x7f82500ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:15:59.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.428+0000 7f40c17e3640 1 -- 172.21.15.5:0/199613345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40bc0d1f60 msgr2=0x7f40bc0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.428+0000 7f40c17e3640 1 --2- 172.21.15.5:0/199613345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40bc0d1f60 0x7f40bc0b4cf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f40bc06c470 tx=0x7f40b402bec0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.428+0000 7f7b3bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b340061e0 0x7f7b3404bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.428+0000 7f8254a62640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82500ce260 0x7f82501602c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f823c00b410 tx=0x7f823c00b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.428+0000 7f7b3bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b340061e0 0x7f7b3404bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35888/0 (socket says 172.21.15.5:35888) 2023-11-24T06:15:59.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.428+0000 7f7b3bfff640 1 -- 172.21.15.5:0/2666177189 learned_addr learned my addr 172.21.15.5:0/2666177189 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:59.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f40c17e3640 1 -- 172.21.15.5:0/199613345 shutdown_connections 2023-11-24T06:15:59.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f40c17e3640 1 --2- 172.21.15.5:0/199613345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40bc0d1f60 0x7f40bc0b4cf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f40c17e3640 1 --2- 172.21.15.5:0/199613345 >> v2:172.21.15.6:3300/0 conn(0x7f40bc15d4d0 0x7f40bc0cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f40c17e3640 1 -- 172.21.15.5:0/199613345 >> 172.21.15.5:0/199613345 conn(0x7f40bc071f90 msgr2=0x7f40bc0723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f824d7fa640 1 -- 172.21.15.5:0/1567013879 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c011830 con 0x7f82500ce260 2023-11-24T06:15:59.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f824d7fa640 1 -- 172.21.15.5:0/1567013879 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823c0119b0 con 0x7f82500ce260 2023-11-24T06:15:59.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f824d7fa640 1 -- 172.21.15.5:0/1567013879 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c011d60 con 0x7f82500ce260 2023-11-24T06:15:59.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f1bf37fe640 1 -- 172.21.15.5:0/1773712705 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c00064ef0 con 0x7f1c0c15d820 2023-11-24T06:15:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.428+0000 7f7b40bda640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b34053780 0x7f7b3404b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f7b3b7fe640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b34006670 0x7f7b3404ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f7b3bfff640 1 -- 172.21.15.5:0/2666177189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b34006670 msgr2=0x7f7b3404ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f7b3bfff640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b34006670 0x7f7b3404ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f7b3bfff640 1 -- 172.21.15.5:0/2666177189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b34053780 msgr2=0x7f7b3404b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f7b3bfff640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b34053780 0x7f7b3404b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f7b3bfff640 1 -- 172.21.15.5:0/2666177189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3c04f730 con 0x7f7b340061e0 2023-11-24T06:15:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f7b40bda640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b34053780 0x7f7b3404b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:15:59.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f40c17e3640 1 -- 172.21.15.5:0/199613345 shutdown_connections 2023-11-24T06:15:59.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f8256cc6640 1 -- 172.21.15.5:0/1567013879 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82501f67c0 con 0x7f82500ce260 2023-11-24T06:15:59.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f8256cc6640 1 -- 172.21.15.5:0/1567013879 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825015f8f0 con 0x7f82500ce260 2023-11-24T06:15:59.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.430+0000 7f7b3bfff640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b340061e0 0x7f7b3404bd30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000c430 tx=0x7f7b3000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.430+0000 7f7f51ce7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f4c15d4d0 0x7f7f4c0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.429+0000 7f40c17e3640 1 -- 172.21.15.5:0/199613345 wait complete. 2023-11-24T06:15:59.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.430+0000 7f7f51ce7640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f4c15d8b0 con 0x7f7f4c0d1f60 2023-11-24T06:15:59.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.430+0000 7f7f51ce7640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f4c0cdfb0 con 0x7f7f4c0ce260 2023-11-24T06:15:59.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.430+0000 7f7f51ce7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f4c0b91b0 con 0x7f7f4c15d4d0 2023-11-24T06:15:59.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.431+0000 7f7f4b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f4c15d4d0 0x7f7f4c0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.431+0000 7f7f4b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7f4c15d4d0 0x7f7f4c0cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35902/0 (socket says 172.21.15.5:35902) 2023-11-24T06:15:59.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.431+0000 7f7f4b7fe640 1 -- 172.21.15.5:0/2372518594 learned_addr learned my addr 172.21.15.5:0/2372518594 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:59.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.432+0000 7f40c17e3640 1 Processor -- start 2023-11-24T06:15:59.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.432+0000 7f8256cc6640 1 -- 172.21.15.5:0/1567013879 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8230005250 con 0x7f82500ce260 2023-11-24T06:15:59.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.433+0000 7f40c17e3640 1 -- start start 2023-11-24T06:15:59.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.431+0000 7f7f4b7fe640 1 -- 172.21.15.5:0/2372518594 >> v1:172.21.15.6:6789/0 conn(0x7f7f4c0ce260 legacy=0x7f7f4c0d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:59.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.431+0000 7f7f4b7fe640 1 -- 172.21.15.5:0/2372518594 >> v1:172.21.15.5:6789/0 conn(0x7f7f4c0d1f60 legacy=0x7f7f4c0b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:15:59.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.431+0000 7f7f4b7fe640 1 -- 172.21.15.5:0/2372518594 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c0b2400 con 0x7f7f4c15d4d0 2023-11-24T06:15:59.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.433+0000 7f7f4b7fe640 1 --2- 172.21.15.5:0/2372518594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c15d4d0 0x7f7f4c0cda70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c06d2a0 tx=0x7f7f3c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7f08455bf5bf9d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.432+0000 7f4e197fa640 1 -- 172.21.15.5:0/3195762394 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4e14005410 con 0x7f4e1c15d4d0 2023-11-24T06:15:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.434+0000 7f824d7fa640 1 -- 172.21.15.5:0/1567013879 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f823c01a400 con 0x7f82500ce260 2023-11-24T06:15:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.434+0000 7f4e197fa640 1 --2- 172.21.15.5:0/3195762394 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4dfc06af10 0x7f4dfc06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.434+0000 7f4e197fa640 1 -- 172.21.15.5:0/3195762394 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e14088ea0 con 0x7f4e1c15d4d0 2023-11-24T06:15:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.435+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/3195762394 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4dfc06af10 0x7f4dfc06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.435+0000 7f40c17e3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40bc0ce260 0x7f40bc14d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.435+0000 7f40c17e3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40bc0d1f60 0x7f40bc1508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.435+0000 7f40c17e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40bc15d4d0 0x7f40bc149800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.435+0000 7f824d7fa640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f823406af10 0x7f823406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.435+0000 7f824d7fa640 1 -- 172.21.15.5:0/1567013879 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f823c089780 con 0x7f82500ce260 2023-11-24T06:15:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.435+0000 7f40c17e3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc153880 con 0x7f40bc15d4d0 2023-11-24T06:15:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.435+0000 7f40baffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40bc0ce260 0x7f40bc14d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.436+0000 7f40baffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40bc0ce260 0x7f40bc14d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:50150/0 (socket says 172.21.15.5:50150) 2023-11-24T06:15:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.436+0000 7f40baffd640 1 -- 172.21.15.5:0/2511801710 learned_addr learned my addr 172.21.15.5:0/2511801710 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.436+0000 7f40c17e3640 1 -- 172.21.15.5:0/2511801710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc150e10 con 0x7f40bc0ce260 2023-11-24T06:15:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.436+0000 7f40bb7fe640 1 --2- 172.21.15.5:0/2511801710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40bc15d4d0 0x7f40bc149800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.436+0000 7f824ffff640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f823406af10 0x7f823406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.436+0000 7f40ba7fc640 1 --2- 172.21.15.5:0/2511801710 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40bc0d1f60 0x7f40bc1508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.436+0000 7f40c17e3640 1 -- 172.21.15.5:0/2511801710 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc153350 con 0x7f40bc0d1f60 2023-11-24T06:15:59.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.436+0000 7f4e1bfff640 1 --2- 172.21.15.5:0/3195762394 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4dfc06af10 0x7f4dfc06d3d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009a00 tx=0x7f4e04008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.437+0000 7f824ffff640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f823406af10 0x7f823406d3d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8250154e40 tx=0x7f8244009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.437+0000 7f40baffd640 1 -- 172.21.15.5:0/2511801710 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40bc0d1f60 msgr2=0x7f40bc1508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.437+0000 7f40baffd640 1 --2- 172.21.15.5:0/2511801710 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40bc0d1f60 0x7f40bc1508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.437+0000 7f40baffd640 1 -- 172.21.15.5:0/2511801710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40bc15d4d0 msgr2=0x7f40bc149800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.437+0000 7f40baffd640 1 --2- 172.21.15.5:0/2511801710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40bc15d4d0 0x7f40bc149800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.437+0000 7f40baffd640 1 -- 172.21.15.5:0/2511801710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b4009a70 con 0x7f40bc0ce260 2023-11-24T06:15:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.438+0000 7f40baffd640 1 --2- 172.21.15.5:0/2511801710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40bc0ce260 0x7f40bc14d480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40a400e8a0 tx=0x7f40a400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.438+0000 7f40b37fe640 1 -- 172.21.15.5:0/2511801710 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a40094f0 con 0x7f40bc0ce260 2023-11-24T06:15:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.438+0000 7f40b37fe640 1 -- 172.21.15.5:0/2511801710 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a400c8e0 con 0x7f40bc0ce260 2023-11-24T06:15:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.438+0000 7f40b37fe640 1 -- 172.21.15.5:0/2511801710 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a400cc70 con 0x7f40bc0ce260 2023-11-24T06:15:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.438+0000 7f40c17e3640 1 -- 172.21.15.5:0/2511801710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc149d40 con 0x7f40bc0ce260 2023-11-24T06:15:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.439+0000 7f40c17e3640 1 -- 172.21.15.5:0/2511801710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc152b60 con 0x7f40bc0ce260 2023-11-24T06:15:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.440+0000 7f4e197fa640 1 -- 172.21.15.5:0/3195762394 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4e1c0c81f0 con 0x7f4e1c15d4d0 2023-11-24T06:15:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.440+0000 7f40c17e3640 1 -- 172.21.15.5:0/2511801710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40bc1529c0 con 0x7f40bc0ce260 2023-11-24T06:15:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.440+0000 7f7b38ff9640 1 -- 172.21.15.5:0/2666177189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b300052d0 con 0x7f7b340061e0 2023-11-24T06:15:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.440+0000 7f7f4a7fc640 1 -- 172.21.15.5:0/2372518594 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c031640 con 0x7f7f4c15d4d0 2023-11-24T06:15:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.441+0000 7f7f4a7fc640 1 -- 172.21.15.5:0/2372518594 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c0317a0 con 0x7f7f4c15d4d0 2023-11-24T06:15:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.441+0000 7f7f4a7fc640 1 -- 172.21.15.5:0/2372518594 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c031b30 con 0x7f7f4c15d4d0 2023-11-24T06:15:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.442+0000 7f7f51ce7640 1 -- 172.21.15.5:0/2372518594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c15d4d0 msgr2=0x7f7f4c0cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.441+0000 7f7b413db640 1 -- 172.21.15.5:0/2666177189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b340da190 con 0x7f7b340061e0 2023-11-24T06:15:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.441+0000 7f7b413db640 1 -- 172.21.15.5:0/2666177189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b340d7a10 con 0x7f7b340061e0 2023-11-24T06:15:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.443+0000 7f7b38ff9640 1 -- 172.21.15.5:0/2666177189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b30005450 con 0x7f7b340061e0 2023-11-24T06:15:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.443+0000 7f7b38ff9640 1 -- 172.21.15.5:0/2666177189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30003830 con 0x7f7b340061e0 2023-11-24T06:15:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.442+0000 7f7f51ce7640 1 --2- 172.21.15.5:0/2372518594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c15d4d0 0x7f7f4c0cda70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c06d2a0 tx=0x7f7f3c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.442+0000 7f7f51ce7640 1 -- 172.21.15.5:0/2372518594 shutdown_connections 2023-11-24T06:15:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.442+0000 7f7f51ce7640 1 --2- 172.21.15.5:0/2372518594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c15d4d0 0x7f7f4c0cda70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.442+0000 7f7f51ce7640 1 -- 172.21.15.5:0/2372518594 >> 172.21.15.5:0/2372518594 conn(0x7f7f4c071f90 msgr2=0x7f7f4c0723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.444+0000 7f7b413db640 1 -- 172.21.15.5:0/2666177189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1c005250 con 0x7f7b340061e0 2023-11-24T06:15:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.444+0000 7f40b37fe640 1 -- 172.21.15.5:0/2511801710 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40a4013470 con 0x7f40bc0ce260 2023-11-24T06:15:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.446+0000 7f40b37fe640 1 --2- 172.21.15.5:0/2511801710 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f409c06ae40 0x7f409c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.446+0000 7f40b37fe640 1 -- 172.21.15.5:0/2511801710 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40a40896f0 con 0x7f40bc0ce260 2023-11-24T06:15:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.447+0000 7f40ba7fc640 1 --2- 172.21.15.5:0/2511801710 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f409c06ae40 0x7f409c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.448+0000 7f7f51ce7640 1 -- 172.21.15.5:0/2372518594 shutdown_connections 2023-11-24T06:15:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.448+0000 7f7f51ce7640 1 -- 172.21.15.5:0/2372518594 wait complete. 2023-11-24T06:15:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.449+0000 7f40ba7fc640 1 --2- 172.21.15.5:0/2511801710 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f409c06ae40 0x7f409c06d300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0c6ad0 tx=0x7f40ac007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.445+0000 7f7b38ff9640 1 -- 172.21.15.5:0/2666177189 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b30003990 con 0x7f7b340061e0 2023-11-24T06:15:59.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.447+0000 7f7b38ff9640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7b2006ae40 0x7f7b2006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.447+0000 7f7b38ff9640 1 -- 172.21.15.5:0/2666177189 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b30088ca0 con 0x7f7b340061e0 2023-11-24T06:15:59.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.450+0000 7f7b3b7fe640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7b2006ae40 0x7f7b2006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.450+0000 7f7f51ce7640 1 Processor -- start 2023-11-24T06:15:59.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.451+0000 7f7f51ce7640 1 -- start start 2023-11-24T06:15:59.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.452+0000 7f7b3b7fe640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7b2006ae40 0x7f7b2006d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b3404bab0 tx=0x7f7b28009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.452+0000 7f7b38ff9640 1 -- 172.21.15.5:0/2666177189 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b30053d40 con 0x7f7b340061e0 2023-11-24T06:15:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.453+0000 7f7f51ce7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c0ce260 0x7f7f4c0c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.453+0000 7f7f51ce7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7f4c0d1f60 0x7f7f4c15ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.453+0000 7f7f51ce7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f4c15d4d0 0x7f7f4c14e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.453+0000 7f7f51ce7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c15f320 con 0x7f7f4c0ce260 2023-11-24T06:15:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.453+0000 7f7f51ce7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c1606e0 con 0x7f7f4c15d4d0 2023-11-24T06:15:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.453+0000 7f7f51ce7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4c0cba40 con 0x7f7f4c0d1f60 2023-11-24T06:15:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.453+0000 7f7f4b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c0ce260 0x7f7f4c0c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.453+0000 7f7f4b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c0ce260 0x7f7f4c0c9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35920/0 (socket says 172.21.15.5:35920) 2023-11-24T06:15:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.453+0000 7f7f4b7fe640 1 -- 172.21.15.5:0/3665135142 learned_addr learned my addr 172.21.15.5:0/3665135142 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:15:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.453+0000 7f7f4affd640 1 --2- 172.21.15.5:0/3665135142 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7f4c0d1f60 0x7f7f4c15ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.453+0000 7f7f4bfff640 1 --2- 172.21.15.5:0/3665135142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f4c15d4d0 0x7f7f4c14e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.454+0000 7f7f4affd640 1 -- 172.21.15.5:0/3665135142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f4c15d4d0 msgr2=0x7f7f4c14e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.454+0000 7f7f4affd640 1 --2- 172.21.15.5:0/3665135142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f4c15d4d0 0x7f7f4c14e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.454+0000 7f7f4affd640 1 -- 172.21.15.5:0/3665135142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c0ce260 msgr2=0x7f7f4c0c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.454+0000 7f7f4affd640 1 --2- 172.21.15.5:0/3665135142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c0ce260 0x7f7f4c0c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.454+0000 7f7f4affd640 1 -- 172.21.15.5:0/3665135142 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f3c0094d0 con 0x7f7f4c0d1f60 2023-11-24T06:15:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.455+0000 7f7f4affd640 1 --2- 172.21.15.5:0/3665135142 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7f4c0d1f60 0x7f7f4c15ede0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800c410 tx=0x7f7f3800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.455+0000 7f7f43fff640 1 -- 172.21.15.5:0/3665135142 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38005230 con 0x7f7f4c0d1f60 2023-11-24T06:15:59.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.455+0000 7f7f43fff640 1 -- 172.21.15.5:0/3665135142 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f380053b0 con 0x7f7f4c0d1f60 2023-11-24T06:15:59.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.456+0000 7f7f51ce7640 1 -- 172.21.15.5:0/3665135142 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4c0cbd20 con 0x7f7f4c0d1f60 2023-11-24T06:15:59.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.456+0000 7f7f51ce7640 1 -- 172.21.15.5:0/3665135142 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4c0cc2e0 con 0x7f7f4c0d1f60 2023-11-24T06:15:59.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.456+0000 7f7f43fff640 1 -- 172.21.15.5:0/3665135142 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38006a10 con 0x7f7f4c0d1f60 2023-11-24T06:15:59.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.456+0000 7f824d7fa640 1 -- 172.21.15.5:0/1567013879 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f823c054810 con 0x7f82500ce260 2023-11-24T06:15:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.458+0000 7f7f51ce7640 1 -- 172.21.15.5:0/3665135142 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4c067960 con 0x7f7f4c0d1f60 2023-11-24T06:15:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.461+0000 7f7f43fff640 1 -- 172.21.15.5:0/3665135142 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7f380045e0 con 0x7f7f4c0d1f60 2023-11-24T06:15:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.462+0000 7f7f43fff640 1 --2- 172.21.15.5:0/3665135142 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7f3406ae40 0x7f7f3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:15:59.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.462+0000 7f7f43fff640 1 -- 172.21.15.5:0/3665135142 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f38088c10 con 0x7f7f4c0d1f60 2023-11-24T06:15:59.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.462+0000 7f7f4b7fe640 1 --2- 172.21.15.5:0/3665135142 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7f3406ae40 0x7f7f3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:15:59.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.464+0000 7f7f4b7fe640 1 --2- 172.21.15.5:0/3665135142 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7f3406ae40 0x7f7f3406d300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c02b930 tx=0x7f7f3c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:15:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.481+0000 7f40b37fe640 1 -- 172.21.15.5:0/2511801710 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40bc1529c0 con 0x7f40bc0ce260 2023-11-24T06:15:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.483+0000 7f7f43fff640 1 -- 172.21.15.5:0/3665135142 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7f38053e00 con 0x7f7f4c0d1f60 2023-11-24T06:15:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.817+0000 7fe9974e0640 1 -- 172.21.15.5:0/3342569425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe9880c78f0 con 0x7fe9880530a0 2023-11-24T06:15:59.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.819+0000 7fe985ffb640 1 -- 172.21.15.5:0/3342569425 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe980017020 con 0x7fe9880530a0 2023-11-24T06:15:59.824 INFO:teuthology.orchestra.run.smithi005.stdout:60129542148 2023-11-24T06:15:59.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.823+0000 7fe9974e0640 1 -- 172.21.15.5:0/3342569425 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe97406af10 msgr2=0x7fe97406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.824+0000 7fe9974e0640 1 --2- 172.21.15.5:0/3342569425 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe97406af10 0x7fe97406d3d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe98c009870 tx=0x7fe98c008040 comp rx=0 tx=0).stop 2023-11-24T06:15:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.824+0000 7fe9974e0640 1 -- 172.21.15.5:0/3342569425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9880530a0 msgr2=0x7fe98804ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.824+0000 7fe9974e0640 1 --2- 172.21.15.5:0/3342569425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9880530a0 0x7fe98804ab20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe98000c450 tx=0x7fe98000c910 comp rx=0 tx=0).stop 2023-11-24T06:15:59.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.825+0000 7fe9974e0640 1 -- 172.21.15.5:0/3342569425 shutdown_connections 2023-11-24T06:15:59.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.825+0000 7fe9974e0640 1 --2- 172.21.15.5:0/3342569425 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe97406af10 0x7fe97406d3d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.825+0000 7fe9974e0640 1 --2- 172.21.15.5:0/3342569425 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe9880df100 0x7fe98804b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.825+0000 7fe9974e0640 1 --2- 172.21.15.5:0/3342569425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9880530a0 0x7fe98804ab20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.825+0000 7fe9974e0640 1 --2- 172.21.15.5:0/3342569425 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe988050180 0x7fe98817ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.825+0000 7fe9974e0640 1 -- 172.21.15.5:0/3342569425 >> 172.21.15.5:0/3342569425 conn(0x7fe988009fb0 msgr2=0x7fe9880de350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.826+0000 7fe9974e0640 1 -- 172.21.15.5:0/3342569425 shutdown_connections 2023-11-24T06:15:59.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.826+0000 7fe9974e0640 1 -- 172.21.15.5:0/3342569425 wait complete. 2023-11-24T06:15:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.834+0000 7f1c113d5640 1 -- 172.21.15.5:0/1773712705 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1bec0057d0 con 0x7f1c0c15d820 2023-11-24T06:15:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.835+0000 7f1bf37fe640 1 -- 172.21.15.5:0/1773712705 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1c00028030 con 0x7f1c0c15d820 2023-11-24T06:15:59.841 INFO:teuthology.orchestra.run.smithi005.stdout:60129542148 2023-11-24T06:15:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.839+0000 7f1c113d5640 1 -- 172.21.15.5:0/1773712705 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1bf806aea0 msgr2=0x7f1bf806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.839+0000 7f1c113d5640 1 --2- 172.21.15.5:0/1773712705 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1bf806aea0 0x7f1bf806d360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0b2090 tx=0x7f1c04007300 comp rx=0 tx=0).stop 2023-11-24T06:15:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.839+0000 7f1c113d5640 1 -- 172.21.15.5:0/1773712705 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c0c15d820 msgr2=0x7f1c0c1514b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.839+0000 7f1c113d5640 1 --2- 172.21.15.5:0/1773712705 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c0c15d820 0x7f1c0c1514b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000c5a0 tx=0x7f1c0000ca60 comp rx=0 tx=0).stop 2023-11-24T06:15:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.840+0000 7f1c113d5640 1 -- 172.21.15.5:0/1773712705 shutdown_connections 2023-11-24T06:15:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.840+0000 7f1c113d5640 1 --2- 172.21.15.5:0/1773712705 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1bf806aea0 0x7f1bf806d360 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.840+0000 7f1c113d5640 1 --2- 172.21.15.5:0/1773712705 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1c0c15d820 0x7f1c0c1514b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.840+0000 7f1c113d5640 1 --2- 172.21.15.5:0/1773712705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c159240 0x7f1c0c14e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.840+0000 7f1c113d5640 1 --2- 172.21.15.5:0/1773712705 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1c0c0ce6a0 0x7f1c0c14fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.840+0000 7f1c113d5640 1 -- 172.21.15.5:0/1773712705 >> 172.21.15.5:0/1773712705 conn(0x7f1c0c0b0840 msgr2=0x7f1c0c0af5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.841+0000 7f1c113d5640 1 -- 172.21.15.5:0/1773712705 shutdown_connections 2023-11-24T06:15:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.841+0000 7f1c113d5640 1 -- 172.21.15.5:0/1773712705 wait complete. 2023-11-24T06:15:59.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.0 2023-11-24T06:15:59.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.855+0000 7fbe725b0640 1 -- 172.21.15.5:0/787532581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbe6404a9a0 con 0x7fbe64053060 2023-11-24T06:15:59.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.855+0000 7f7b413db640 1 -- 172.21.15.5:0/2666177189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f7b1c005500 con 0x7f7b340061e0 2023-11-24T06:15:59.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.856+0000 7fbe68ff9640 1 -- 172.21.15.5:0/787532581 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbe6404a9a0 con 0x7fbe64053060 2023-11-24T06:15:59.861 INFO:teuthology.orchestra.run.smithi005.stdout:60129542149 2023-11-24T06:15:59.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.856+0000 7f7b38ff9640 1 -- 172.21.15.5:0/2666177189 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b30017090 con 0x7f7b340061e0 2023-11-24T06:15:59.862 INFO:teuthology.orchestra.run.smithi005.stdout:60129542148 2023-11-24T06:15:59.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.858+0000 7fbe725b0640 1 -- 172.21.15.5:0/787532581 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbe5006ae40 msgr2=0x7fbe5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.858+0000 7fbe725b0640 1 --2- 172.21.15.5:0/787532581 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbe5006ae40 0x7fbe5006d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe60037000 tx=0x7fbe60031000 comp rx=0 tx=0).stop 2023-11-24T06:15:59.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.858+0000 7fbe725b0640 1 -- 172.21.15.5:0/787532581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe64053060 msgr2=0x7fbe640cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.858+0000 7fbe725b0640 1 --2- 172.21.15.5:0/787532581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe64053060 0x7fbe640cc6b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c400 tx=0x7fbe5800c8c0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.859+0000 7fbe725b0640 1 -- 172.21.15.5:0/787532581 shutdown_connections 2023-11-24T06:15:59.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.859+0000 7fbe725b0640 1 --2- 172.21.15.5:0/787532581 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbe5006ae40 0x7fbe5006d300 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe60037000 tx=0x7fbe60031000 comp rx=0 tx=0).stop 2023-11-24T06:15:59.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.859+0000 7fbe725b0640 1 --2- 172.21.15.5:0/787532581 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe640df0c0 0x7fbe6404ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.859+0000 7fbe725b0640 1 --2- 172.21.15.5:0/787532581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe64053060 0x7fbe640cc6b0 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800c400 tx=0x7fbe5800c8c0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.859+0000 7fbe725b0640 1 --2- 172.21.15.5:0/787532581 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe64050140 0x7fbe64178b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.859+0000 7fbe725b0640 1 -- 172.21.15.5:0/787532581 >> 172.21.15.5:0/787532581 conn(0x7fbe64009f70 msgr2=0x7fbe640071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.859+0000 7f7b413db640 1 -- 172.21.15.5:0/2666177189 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7b2006ae40 msgr2=0x7f7b2006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.859+0000 7f7b413db640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7b2006ae40 0x7f7b2006d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b3404bab0 tx=0x7f7b28009040 comp rx=0 tx=0).stop 2023-11-24T06:15:59.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.859+0000 7fbe725b0640 1 -- 172.21.15.5:0/787532581 shutdown_connections 2023-11-24T06:15:59.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.860+0000 7fbe725b0640 1 -- 172.21.15.5:0/787532581 wait complete. 2023-11-24T06:15:59.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.859+0000 7f7b413db640 1 -- 172.21.15.5:0/2666177189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b340061e0 msgr2=0x7f7b3404bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.859+0000 7f7b413db640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b340061e0 0x7f7b3404bd30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000c430 tx=0x7f7b3000c8f0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.861+0000 7f7b413db640 1 -- 172.21.15.5:0/2666177189 shutdown_connections 2023-11-24T06:15:59.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.861+0000 7f7b413db640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7b2006ae40 0x7f7b2006d300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.861+0000 7f7b413db640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7b34006670 0x7f7b3404ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.861+0000 7f7b413db640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7b34053780 0x7f7b3404b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.861+0000 7f7b413db640 1 --2- 172.21.15.5:0/2666177189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b340061e0 0x7f7b3404bd30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.861+0000 7f7b413db640 1 -- 172.21.15.5:0/2666177189 >> 172.21.15.5:0/2666177189 conn(0x7f7b3403aa10 msgr2=0x7f7b3403c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.861+0000 7f7b413db640 1 -- 172.21.15.5:0/2666177189 shutdown_connections 2023-11-24T06:15:59.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.861+0000 7f7b413db640 1 -- 172.21.15.5:0/2666177189 wait complete. 2023-11-24T06:15:59.872 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.1 2023-11-24T06:15:59.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.874+0000 7f4e22333640 1 -- 172.21.15.5:0/3195762394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4e1c0c81f0 con 0x7f4e1c15d4d0 2023-11-24T06:15:59.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.875+0000 7f4e197fa640 1 -- 172.21.15.5:0/3195762394 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4e1c0c81f0 con 0x7f4e1c15d4d0 2023-11-24T06:15:59.879 INFO:teuthology.orchestra.run.smithi005.stdout:60129542148 2023-11-24T06:15:59.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.879+0000 7f4e22333640 1 -- 172.21.15.5:0/3195762394 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4dfc06af10 msgr2=0x7f4dfc06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.879+0000 7f4e22333640 1 --2- 172.21.15.5:0/3195762394 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4dfc06af10 0x7f4dfc06d3d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009a00 tx=0x7f4e04008040 comp rx=0 tx=0).stop 2023-11-24T06:15:59.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.879+0000 7f4e22333640 1 -- 172.21.15.5:0/3195762394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c15d4d0 msgr2=0x7f4e1c1f2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.879+0000 7f4e22333640 1 --2- 172.21.15.5:0/3195762394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c15d4d0 0x7f4e1c1f2a60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4e1400c7b0 tx=0x7f4e1400cc70 comp rx=0 tx=0).stop 2023-11-24T06:15:59.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.879+0000 7f4e22333640 1 -- 172.21.15.5:0/3195762394 shutdown_connections 2023-11-24T06:15:59.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.879+0000 7f4e22333640 1 --2- 172.21.15.5:0/3195762394 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4dfc06af10 0x7f4dfc06d3d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.879+0000 7f4e22333640 1 --2- 172.21.15.5:0/3195762394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e1c15d4d0 0x7f4e1c1f2a60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.879+0000 7f4e22333640 1 --2- 172.21.15.5:0/3195762394 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4e1c0ce6c0 0x7f4e1c0b7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.879+0000 7f4e22333640 1 --2- 172.21.15.5:0/3195762394 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4e1c0ce260 0x7f4e1c0c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.879+0000 7f4e22333640 1 -- 172.21.15.5:0/3195762394 >> 172.21.15.5:0/3195762394 conn(0x7f4e1c071f90 msgr2=0x7f4e1c15c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.879+0000 7f4e22333640 1 -- 172.21.15.5:0/3195762394 shutdown_connections 2023-11-24T06:15:59.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.879+0000 7f4e22333640 1 -- 172.21.15.5:0/3195762394 wait complete. 2023-11-24T06:15:59.886 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.2 2023-11-24T06:15:59.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2023-11-24T06:15:59.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.889+0000 7f8256cc6640 1 -- 172.21.15.5:0/1567013879 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8230005500 con 0x7f82500ce260 2023-11-24T06:15:59.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.890+0000 7f824d7fa640 1 -- 172.21.15.5:0/1567013879 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f823c057e30 con 0x7f82500ce260 2023-11-24T06:15:59.893 INFO:teuthology.orchestra.run.smithi005.stdout:64424509445 2023-11-24T06:15:59.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.891+0000 7f7f51ce7640 1 -- 172.21.15.5:0/3665135142 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7f4c074770 con 0x7f7f4c0d1f60 2023-11-24T06:15:59.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.891+0000 7f7f43fff640 1 -- 172.21.15.5:0/3665135142 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7f38057420 con 0x7f7f4c0d1f60 2023-11-24T06:15:59.894 INFO:teuthology.orchestra.run.smithi005.stdout:64424509444 2023-11-24T06:15:59.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.892+0000 7f8256cc6640 1 -- 172.21.15.5:0/1567013879 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f823406af10 msgr2=0x7f823406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.892+0000 7f8256cc6640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f823406af10 0x7f823406d3d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8250154e40 tx=0x7f8244009040 comp rx=0 tx=0).stop 2023-11-24T06:15:59.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.892+0000 7f8256cc6640 1 -- 172.21.15.5:0/1567013879 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82500ce260 msgr2=0x7f82501602c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.892+0000 7f8256cc6640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82500ce260 0x7f82501602c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f823c00b410 tx=0x7f823c00b8d0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.893+0000 7f8256cc6640 1 -- 172.21.15.5:0/1567013879 shutdown_connections 2023-11-24T06:15:59.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.893+0000 7f8256cc6640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f823406af10 0x7f823406d3d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.893+0000 7f8256cc6640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f825015d4d0 0x7f8250153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.893+0000 7f8256cc6640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82500d1f60 0x7f82500ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.893+0000 7f8256cc6640 1 --2- 172.21.15.5:0/1567013879 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82500ce260 0x7f82501602c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.893+0000 7f8256cc6640 1 -- 172.21.15.5:0/1567013879 >> 172.21.15.5:0/1567013879 conn(0x7f8250071f90 msgr2=0x7f82500b8310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.893+0000 7f8256cc6640 1 -- 172.21.15.5:0/1567013879 shutdown_connections 2023-11-24T06:15:59.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.893+0000 7f8256cc6640 1 -- 172.21.15.5:0/1567013879 wait complete. 2023-11-24T06:15:59.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.894+0000 7f7f51ce7640 1 -- 172.21.15.5:0/3665135142 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7f3406ae40 msgr2=0x7f7f3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.894+0000 7f7f51ce7640 1 --2- 172.21.15.5:0/3665135142 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7f3406ae40 0x7f7f3406d300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c02b930 tx=0x7f7f3c008030 comp rx=0 tx=0).stop 2023-11-24T06:15:59.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.894+0000 7f7f51ce7640 1 -- 172.21.15.5:0/3665135142 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7f4c0d1f60 msgr2=0x7f7f4c15ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.894+0000 7f7f51ce7640 1 --2- 172.21.15.5:0/3665135142 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7f4c0d1f60 0x7f7f4c15ede0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800c410 tx=0x7f7f3800c8d0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.894+0000 7f7f51ce7640 1 -- 172.21.15.5:0/3665135142 shutdown_connections 2023-11-24T06:15:59.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.894+0000 7f7f51ce7640 1 --2- 172.21.15.5:0/3665135142 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7f3406ae40 0x7f7f3406d300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.894+0000 7f7f51ce7640 1 --2- 172.21.15.5:0/3665135142 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7f4c0d1f60 0x7f7f4c15ede0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.894+0000 7f7f51ce7640 1 --2- 172.21.15.5:0/3665135142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7f4c15d4d0 0x7f7f4c14e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.894+0000 7f7f51ce7640 1 --2- 172.21.15.5:0/3665135142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c0ce260 0x7f7f4c0c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.894+0000 7f7f51ce7640 1 -- 172.21.15.5:0/3665135142 >> 172.21.15.5:0/3665135142 conn(0x7f7f4c071f90 msgr2=0x7f7f4c0b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.894+0000 7f7f51ce7640 1 -- 172.21.15.5:0/3665135142 shutdown_connections 2023-11-24T06:15:59.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.894+0000 7f7f51ce7640 1 -- 172.21.15.5:0/3665135142 wait complete. 2023-11-24T06:15:59.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.895+0000 7f40c17e3640 1 -- 172.21.15.5:0/2511801710 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f40bc1529c0 con 0x7f40bc0ce260 2023-11-24T06:15:59.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.896+0000 7f40b37fe640 1 -- 172.21.15.5:0/2511801710 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f40bc1529c0 con 0x7f40bc0ce260 2023-11-24T06:15:59.900 INFO:teuthology.orchestra.run.smithi005.stdout:60129542150 2023-11-24T06:15:59.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.900+0000 7f40c17e3640 1 -- 172.21.15.5:0/2511801710 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f409c06ae40 msgr2=0x7f409c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.900+0000 7f40c17e3640 1 --2- 172.21.15.5:0/2511801710 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f409c06ae40 0x7f409c06d300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0c6ad0 tx=0x7f40ac007e80 comp rx=0 tx=0).stop 2023-11-24T06:15:59.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.900+0000 7f40c17e3640 1 -- 172.21.15.5:0/2511801710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40bc0ce260 msgr2=0x7f40bc14d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:15:59.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.900+0000 7f40c17e3640 1 --2- 172.21.15.5:0/2511801710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40bc0ce260 0x7f40bc14d480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f40a400e8a0 tx=0x7f40a400ed60 comp rx=0 tx=0).stop 2023-11-24T06:15:59.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.900+0000 7f40c17e3640 1 -- 172.21.15.5:0/2511801710 shutdown_connections 2023-11-24T06:15:59.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.900+0000 7f40c17e3640 1 --2- 172.21.15.5:0/2511801710 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f409c06ae40 0x7f409c06d300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.900+0000 7f40c17e3640 1 --2- 172.21.15.5:0/2511801710 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40bc0d1f60 0x7f40bc1508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.900+0000 7f40c17e3640 1 --2- 172.21.15.5:0/2511801710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40bc15d4d0 0x7f40bc149800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.900+0000 7f40c17e3640 1 --2- 172.21.15.5:0/2511801710 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f40bc0ce260 0x7f40bc14d480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:15:59.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.900+0000 7f40c17e3640 1 -- 172.21.15.5:0/2511801710 >> 172.21.15.5:0/2511801710 conn(0x7f40bc071f90 msgr2=0x7f40bc158fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:15:59.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.900+0000 7f40c17e3640 1 -- 172.21.15.5:0/2511801710 shutdown_connections 2023-11-24T06:15:59.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:15:59.900+0000 7f40c17e3640 1 -- 172.21.15.5:0/2511801710 wait complete. 2023-11-24T06:15:59.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.3 2023-11-24T06:15:59.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2023-11-24T06:15:59.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.7 2023-11-24T06:15:59.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.5 2023-11-24T06:16:00.858 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T06:16:00.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T06:16:00.887 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T06:16:00.893 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T06:16:00.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T06:16:00.916 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T06:16:00.916 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T06:16:00.920 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T06:16:01.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.150+0000 7f7ab8a0d640 1 Processor -- start 2023-11-24T06:16:01.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.150+0000 7f7ab8a0d640 1 -- start start 2023-11-24T06:16:01.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.153+0000 7f7ab8a0d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7aac0544f0 0x7f7aac104910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.153+0000 7f7ab8a0d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac100ee0 0x7f7aac104e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.153+0000 7f7ab8a0d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7aac105390 0x7f7aac0063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.153+0000 7f7ab8a0d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aac0340f0 con 0x7f7aac0544f0 2023-11-24T06:16:01.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.154+0000 7f7ab8a0d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aac037b10 con 0x7f7aac105390 2023-11-24T06:16:01.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.154+0000 7f7ab8a0d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac107c20 con 0x7f7aac100ee0 2023-11-24T06:16:01.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.154+0000 7f7ab2d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7aac0544f0 0x7f7aac104910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.154+0000 7f7ab2d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7aac0544f0 0x7f7aac104910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35932/0 (socket says 172.21.15.5:35932) 2023-11-24T06:16:01.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.154+0000 7f7ab2d9d640 1 -- 172.21.15.5:0/386995376 learned_addr learned my addr 172.21.15.5:0/386995376 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:01.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.154+0000 7f7ab359e640 1 --2- 172.21.15.5:0/386995376 >> v2:172.21.15.6:3300/0 conn(0x7f7aac105390 0x7f7aac0063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.155+0000 7f7ab2d9d640 1 -- 172.21.15.5:0/386995376 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac100ee0 msgr2=0x7f7aac104e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:16:01.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.155+0000 7f7ab2d9d640 1 --2- 172.21.15.5:0/386995376 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac100ee0 0x7f7aac104e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.155+0000 7f7ab2d9d640 1 -- 172.21.15.5:0/386995376 >> v2:172.21.15.6:3300/0 conn(0x7f7aac105390 msgr2=0x7f7aac0063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:01.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.155+0000 7f7ab2d9d640 1 --2- 172.21.15.5:0/386995376 >> v2:172.21.15.6:3300/0 conn(0x7f7aac105390 0x7f7aac0063b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.155+0000 7f7ab2d9d640 1 -- 172.21.15.5:0/386995376 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac1057e0 con 0x7f7aac0544f0 2023-11-24T06:16:01.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.157+0000 7f7ab2d9d640 1 --2- 172.21.15.5:0/386995376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aac0544f0 0x7f7aac104910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40691e0 tx=0x7f7ab4094f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1235c914c9b000b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.158+0000 7f7ab1d9b640 1 -- 172.21.15.5:0/386995376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4095db0 con 0x7f7aac0544f0 2023-11-24T06:16:01.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.158+0000 7f7ab1d9b640 1 -- 172.21.15.5:0/386995376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab409d440 con 0x7f7aac0544f0 2023-11-24T06:16:01.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.158+0000 7f7ab1d9b640 1 -- 172.21.15.5:0/386995376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab409d870 con 0x7f7aac0544f0 2023-11-24T06:16:01.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.159+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/386995376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aac0544f0 msgr2=0x7f7aac104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.159+0000 7f7ab8a0d640 1 --2- 172.21.15.5:0/386995376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aac0544f0 0x7f7aac104910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40691e0 tx=0x7f7ab4094f40 comp rx=0 tx=0).stop 2023-11-24T06:16:01.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.160+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/386995376 shutdown_connections 2023-11-24T06:16:01.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.160+0000 7f7ab8a0d640 1 --2- 172.21.15.5:0/386995376 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac100ee0 0x7f7aac104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.160+0000 7f7ab8a0d640 1 --2- 172.21.15.5:0/386995376 >> v2:172.21.15.6:3300/0 conn(0x7f7aac105390 0x7f7aac0063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.160+0000 7f7ab8a0d640 1 --2- 172.21.15.5:0/386995376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aac0544f0 0x7f7aac104910 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.160+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/386995376 >> 172.21.15.5:0/386995376 conn(0x7f7aac009f80 msgr2=0x7f7aac00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.161+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/386995376 shutdown_connections 2023-11-24T06:16:01.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.162+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/386995376 wait complete. 2023-11-24T06:16:01.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.164+0000 7f7ab8a0d640 1 Processor -- start 2023-11-24T06:16:01.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.165+0000 7f7ab8a0d640 1 -- start start 2023-11-24T06:16:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.167+0000 7f7ab8a0d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aac0544f0 0x7f7aac0f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.167+0000 7f7ab8a0d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac100ee0 0x7f7aac036220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.168+0000 7f7ab8a0d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aac105390 0x7f7aac0485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.168+0000 7f7ab2d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aac0544f0 0x7f7aac0f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.168+0000 7f7ab2d9d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aac0544f0 0x7f7aac0f3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:50184/0 (socket says 172.21.15.5:50184) 2023-11-24T06:16:01.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.168+0000 7f7ab259c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac100ee0 0x7f7aac036220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.168+0000 7f7ab259c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac100ee0 0x7f7aac036220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:53488/0 (socket says 172.21.15.5:53488) 2023-11-24T06:16:01.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.169+0000 7f7ab359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aac105390 0x7f7aac0485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.169+0000 7f7ab359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aac105390 0x7f7aac0485c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35934/0 (socket says 172.21.15.5:35934) 2023-11-24T06:16:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.170+0000 7f7ab2d9d640 1 -- 172.21.15.5:0/556751001 learned_addr learned my addr 172.21.15.5:0/556751001 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.170+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/556751001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac0f7980 con 0x7f7aac105390 2023-11-24T06:16:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.170+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/556751001 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac0f4000 con 0x7f7aac0544f0 2023-11-24T06:16:01.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.170+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/556751001 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac036760 con 0x7f7aac100ee0 2023-11-24T06:16:01.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.171+0000 7f7ab359e640 1 -- 172.21.15.5:0/556751001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac100ee0 msgr2=0x7f7aac036220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.171+0000 7f7ab359e640 1 --2- 172.21.15.5:0/556751001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac100ee0 0x7f7aac036220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.171+0000 7f7ab359e640 1 -- 172.21.15.5:0/556751001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aac0544f0 msgr2=0x7f7aac0f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.171+0000 7f7ab359e640 1 --2- 172.21.15.5:0/556751001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aac0544f0 0x7f7aac0f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.171+0000 7f7ab359e640 1 -- 172.21.15.5:0/556751001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab404f730 con 0x7f7aac105390 2023-11-24T06:16:01.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.172+0000 7f7ab359e640 1 --2- 172.21.15.5:0/556751001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aac105390 0x7f7aac0485c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800eb60 tx=0x7f7aa800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.173+0000 7f7ab2d9d640 1 --2- 172.21.15.5:0/556751001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aac0544f0 0x7f7aac0f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:01.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.173+0000 7f7a9b7fe640 1 -- 172.21.15.5:0/556751001 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800cc30 con 0x7f7aac105390 2023-11-24T06:16:01.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.173+0000 7f7a9b7fe640 1 -- 172.21.15.5:0/556751001 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa800cd90 con 0x7f7aac105390 2023-11-24T06:16:01.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.173+0000 7f7a9b7fe640 1 -- 172.21.15.5:0/556751001 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800ee90 con 0x7f7aac105390 2023-11-24T06:16:01.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.174+0000 7f7ab259c640 1 --2- 172.21.15.5:0/556751001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac100ee0 0x7f7aac036220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:01.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.174+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/556751001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aac036ca0 con 0x7f7aac105390 2023-11-24T06:16:01.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.174+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/556751001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aac04c5b0 con 0x7f7aac105390 2023-11-24T06:16:01.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.181+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/556751001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aac005290 con 0x7f7aac105390 2023-11-24T06:16:01.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.181+0000 7f7a9b7fe640 1 -- 172.21.15.5:0/556751001 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7aa8013470 con 0x7f7aac105390 2023-11-24T06:16:01.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.183+0000 7f7a9b7fe640 1 --2- 172.21.15.5:0/556751001 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7aa006af40 0x7f7aa006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.187+0000 7f7ab2d9d640 1 --2- 172.21.15.5:0/556751001 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7aa006af40 0x7f7aa006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.187+0000 7f7a9b7fe640 1 -- 172.21.15.5:0/556751001 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7aa80896a0 con 0x7f7aac105390 2023-11-24T06:16:01.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.192+0000 7f7ab2d9d640 1 --2- 172.21.15.5:0/556751001 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7aa006af40 0x7f7aa006d400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40753a0 tx=0x7f7ab4099000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.193+0000 7f7a9b7fe640 1 -- 172.21.15.5:0/556751001 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7aa80547c0 con 0x7f7aac105390 2023-11-24T06:16:01.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.199+0000 7f8c4d49f640 1 Processor -- start 2023-11-24T06:16:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.200+0000 7f0dd2d2e640 1 Processor -- start 2023-11-24T06:16:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.199+0000 7f8c4d49f640 1 -- start start 2023-11-24T06:16:01.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.200+0000 7f0dd2d2e640 1 -- start start 2023-11-24T06:16:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.202+0000 7f0dd2d2e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0dcc0d1d80 0x7f0dcc0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.202+0000 7f0dd2d2e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dcc0d04b0 con 0x7f0dcc15d5d0 2023-11-24T06:16:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.202+0000 7f0dd2d2e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dcc0cdb80 con 0x7f0dcc0cf710 2023-11-24T06:16:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.202+0000 7f0dd2d2e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc0ce4e0 con 0x7f0dcc0d1d80 2023-11-24T06:16:01.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.202+0000 7f8c4d49f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c480cf710 0x7f8c480cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.202+0000 7f8c4d49f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c480d1670 0x7f8c480d1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.202+0000 7f8c4d49f640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c480d04b0 con 0x7f8c4815d5d0 2023-11-24T06:16:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.202+0000 7f8c4d49f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c480ce4e0 con 0x7f8c480d1670 2023-11-24T06:16:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.202+0000 7f8c4d49f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480cf010 con 0x7f8c480cf710 2023-11-24T06:16:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.203+0000 7f0dd1d2c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0dcc15d5d0 0x7f0dcc0ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56598/0 (socket says 172.21.15.5:56598) 2023-11-24T06:16:01.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.203+0000 7f0dd1d2c640 1 -- 172.21.15.5:0/793396295 learned_addr learned my addr 172.21.15.5:0/793396295 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.203+0000 7f8c477fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c480cf710 0x7f8c480cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f8c477fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c480cf710 0x7f8c480cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:53504/0 (socket says 172.21.15.5:53504) 2023-11-24T06:16:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f8c477fe640 1 -- 172.21.15.5:0/206882985 learned_addr learned my addr 172.21.15.5:0/206882985 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:01.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f0dd0d2a640 1 -- 172.21.15.5:0/793396295 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886788558 0 0) 0x7f0dcc0d04b0 con 0x7f0dcc15d5d0 2023-11-24T06:16:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f8c4cc9e640 1 --2- 172.21.15.5:0/206882985 >> v2:172.21.15.5:3300/0 conn(0x7f8c480d1670 0x7f8c480d1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f0dd0d2a640 1 -- 172.21.15.5:0/793396295 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dbc003520 con 0x7f0dcc15d5d0 2023-11-24T06:16:01.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f8c477fe640 1 -- 172.21.15.5:0/206882985 >> v2:172.21.15.5:3300/0 conn(0x7f8c480d1670 msgr2=0x7f8c480d1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f0dd0d2a640 1 -- 172.21.15.5:0/793396295 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2469553282 0 0) 0x7f0dbc003520 con 0x7f0dcc15d5d0 2023-11-24T06:16:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f0dd0d2a640 1 -- 172.21.15.5:0/793396295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0dcc0d1d80 msgr2=0x7f0dcc0cfb70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T06:16:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f0dd0d2a640 1 --2- 172.21.15.5:0/793396295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0dcc0d1d80 0x7f0dcc0cfb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f0dd0d2a640 1 -- 172.21.15.5:0/793396295 >> v1:172.21.15.6:6789/0 conn(0x7f0dcc0cf710 legacy=0x7f0dcc0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f0dd0d2a640 1 -- 172.21.15.5:0/793396295 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc0d01d0 con 0x7f0dcc15d5d0 2023-11-24T06:16:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f0dd0d2a640 1 -- 172.21.15.5:0/793396295 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f0dc4001670 con 0x7f0dcc15d5d0 2023-11-24T06:16:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f0dd0d2a640 1 -- 172.21.15.5:0/793396295 >> v1:172.21.15.5:6789/0 conn(0x7f0dcc15d5d0 legacy=0x7f0dcc0ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f0dd0d2a640 1 --2- 172.21.15.5:0/793396295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0dbc003a10 0x7f0dbc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f0dd0d2a640 1 --2- 172.21.15.5:0/793396295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dbc0047c0 0x7f0dbc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f0dd0d2a640 1 --2- 172.21.15.5:0/793396295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0dbc005330 0x7f0dbc014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f0dd0d2a640 1 -- 172.21.15.5:0/793396295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc0ce4e0 con 0x7f0dbc0047c0 2023-11-24T06:16:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f0dd0d2a640 1 -- 172.21.15.5:0/793396295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc014a00 con 0x7f0dbc003a10 2023-11-24T06:16:01.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.205+0000 7f0dd0d2a640 1 -- 172.21.15.5:0/793396295 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc014bc0 con 0x7f0dbc005330 2023-11-24T06:16:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f8c477fe640 1 --2- 172.21.15.5:0/206882985 >> v2:172.21.15.5:3300/0 conn(0x7f8c480d1670 0x7f8c480d1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f8c477fe640 1 -- 172.21.15.5:0/206882985 >> v1:172.21.15.6:6789/0 conn(0x7f8c4815d5d0 legacy=0x7f8c480ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.204+0000 7f8c477fe640 1 -- 172.21.15.5:0/206882985 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c480cdb80 con 0x7f8c480cf710 2023-11-24T06:16:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.206+0000 7f0dd1d2c640 1 --2- 172.21.15.5:0/793396295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0dbc003a10 0x7f0dbc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.206+0000 7f0dd152b640 1 --2- 172.21.15.5:0/793396295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dbc0047c0 0x7f0dbc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.206+0000 7f8c477fe640 1 --2- 172.21.15.5:0/206882985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c480cf710 0x7f8c480cfb70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009870 tx=0x7f8c3802b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74c0c9ee67112a72 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.206+0000 7f0dd1d2c640 1 -- 172.21.15.5:0/793396295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0dbc005330 msgr2=0x7f0dbc014190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:01.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.206+0000 7f0dd252d640 1 --2- 172.21.15.5:0/793396295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0dbc005330 0x7f0dbc014190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.207+0000 7f0dd1d2c640 1 --2- 172.21.15.5:0/793396295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0dbc005330 0x7f0dbc014190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.207+0000 7f0dd1d2c640 1 -- 172.21.15.5:0/793396295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dbc0047c0 msgr2=0x7f0dbc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.207+0000 7f0dd1d2c640 1 --2- 172.21.15.5:0/793396295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dbc0047c0 0x7f0dbc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.207+0000 7f0dd1d2c640 1 -- 172.21.15.5:0/793396295 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc0d01d0 con 0x7f0dbc003a10 2023-11-24T06:16:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.207+0000 7f0dd152b640 1 --2- 172.21.15.5:0/793396295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dbc0047c0 0x7f0dbc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.208+0000 7f8c46ffd640 1 -- 172.21.15.5:0/206882985 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c380317c0 con 0x7f8c480cf710 2023-11-24T06:16:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.208+0000 7f8c46ffd640 1 -- 172.21.15.5:0/206882985 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c38031920 con 0x7f8c480cf710 2023-11-24T06:16:01.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.208+0000 7f8c46ffd640 1 -- 172.21.15.5:0/206882985 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c38031cb0 con 0x7f8c480cf710 2023-11-24T06:16:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.208+0000 7f0dd1d2c640 1 --2- 172.21.15.5:0/793396295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0dbc003a10 0x7f0dbc003e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400c290 tx=0x7f0dbc0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a456858c475bf43a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.209+0000 7f8c4d49f640 1 -- 172.21.15.5:0/206882985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c480cf710 msgr2=0x7f8c480cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.209+0000 7f8c4d49f640 1 --2- 172.21.15.5:0/206882985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c480cf710 0x7f8c480cfb70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009870 tx=0x7f8c3802b860 comp rx=0 tx=0).stop 2023-11-24T06:16:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.209+0000 7f0dd0d2a640 1 -- 172.21.15.5:0/793396295 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc402cb40 con 0x7f0dbc003a10 2023-11-24T06:16:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.209+0000 7f0dd0d2a640 1 -- 172.21.15.5:0/793396295 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc402cca0 con 0x7f0dbc003a10 2023-11-24T06:16:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.209+0000 7f0dd0d2a640 1 -- 172.21.15.5:0/793396295 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4005b10 con 0x7f0dbc003a10 2023-11-24T06:16:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.209+0000 7f8c4d49f640 1 -- 172.21.15.5:0/206882985 shutdown_connections 2023-11-24T06:16:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.209+0000 7f8c4d49f640 1 --2- 172.21.15.5:0/206882985 >> v2:172.21.15.5:3300/0 conn(0x7f8c480d1670 0x7f8c480d1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.209+0000 7f8c4d49f640 1 --2- 172.21.15.5:0/206882985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c480cf710 0x7f8c480cfb70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.209+0000 7f8c4d49f640 1 -- 172.21.15.5:0/206882985 >> 172.21.15.5:0/206882985 conn(0x7f8c4809c3d0 msgr2=0x7f8c480d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.210+0000 7f8c4d49f640 1 -- 172.21.15.5:0/206882985 shutdown_connections 2023-11-24T06:16:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.210+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/793396295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0dbc003a10 msgr2=0x7f0dbc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.210+0000 7f0dd2d2e640 1 --2- 172.21.15.5:0/793396295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0dbc003a10 0x7f0dbc003e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400c290 tx=0x7f0dbc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.210+0000 7f0dd252d640 1 -- 172.21.15.5:0/793396295 reap_dead start 2023-11-24T06:16:01.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.210+0000 7f8c4d49f640 1 -- 172.21.15.5:0/206882985 wait complete. 2023-11-24T06:16:01.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.210+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/793396295 shutdown_connections 2023-11-24T06:16:01.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.211+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/793396295 >> 172.21.15.5:0/793396295 conn(0x7f0dcc09c3d0 msgr2=0x7f0dcc0d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.211+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/793396295 shutdown_connections 2023-11-24T06:16:01.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.212+0000 7f4c29a1c640 1 Processor -- start 2023-11-24T06:16:01.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.212+0000 7f4c29a1c640 1 -- start start 2023-11-24T06:16:01.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.212+0000 7f8c4d49f640 1 Processor -- start 2023-11-24T06:16:01.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.211+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/793396295 wait complete. 2023-11-24T06:16:01.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.214+0000 7f0dd2d2e640 1 Processor -- start 2023-11-24T06:16:01.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.214+0000 7f0dd2d2e640 1 -- start start 2023-11-24T06:16:01.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.214+0000 7f4c29a1c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4c240ad720 0x7f4c240b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.214+0000 7f4c29a1c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c240abff0 0x7f4c240b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.214+0000 7f4c29a1c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2415c360 con 0x7f4c2406d520 2023-11-24T06:16:01.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.214+0000 7f4c29a1c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c240b41a0 con 0x7f4c240ad720 2023-11-24T06:16:01.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.214+0000 7f4c29a1c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2415ba30 con 0x7f4c240abff0 2023-11-24T06:16:01.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.213+0000 7f8c4d49f640 1 -- start start 2023-11-24T06:16:01.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.214+0000 7f8c4d49f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c480cf710 0x7f8c4815c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.214+0000 7f8c4d49f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c480d1670 0x7f8c4814f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.214+0000 7f8c4d49f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c4815d5d0 0x7f8c4814fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.214+0000 7f8c4d49f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480cc2e0 con 0x7f8c4815d5d0 2023-11-24T06:16:01.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.215+0000 7f8c4d49f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4815c880 con 0x7f8c480d1670 2023-11-24T06:16:01.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.215+0000 7f8c4d49f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480cb6d0 con 0x7f8c480cf710 2023-11-24T06:16:01.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.215+0000 7f4c237fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c240abff0 0x7f4c240b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.215+0000 7f4c237fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c240abff0 0x7f4c240b25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:53518/0 (socket says 172.21.15.5:53518) 2023-11-24T06:16:01.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.215+0000 7f8c4cc9e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c4815d5d0 0x7f8c4814fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.215+0000 7f4c237fe640 1 -- 172.21.15.5:0/4258316338 learned_addr learned my addr 172.21.15.5:0/4258316338 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.216+0000 7fe583890640 1 Processor -- start 2023-11-24T06:16:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.216+0000 7f4c227fc640 1 --2- 172.21.15.5:0/4258316338 >> v2:172.21.15.6:3300/0 conn(0x7f4c240ad720 0x7f4c240b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.216+0000 7f4c237fe640 1 -- 172.21.15.5:0/4258316338 >> v2:172.21.15.6:3300/0 conn(0x7f4c240ad720 msgr2=0x7f4c240b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:01.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.216+0000 7f4c237fe640 1 --2- 172.21.15.5:0/4258316338 >> v2:172.21.15.6:3300/0 conn(0x7f4c240ad720 0x7f4c240b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.216+0000 7f4c237fe640 1 -- 172.21.15.5:0/4258316338 >> v1:172.21.15.6:6789/0 conn(0x7f4c2406d520 legacy=0x7f4c2406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.216+0000 7f4c237fe640 1 -- 172.21.15.5:0/4258316338 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2415bd10 con 0x7f4c240abff0 2023-11-24T06:16:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.216+0000 7f8c4cc9e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c4815d5d0 0x7f8c4814fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35952/0 (socket says 172.21.15.5:35952) 2023-11-24T06:16:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.216+0000 7f8c4cc9e640 1 -- 172.21.15.5:0/3682346132 learned_addr learned my addr 172.21.15.5:0/3682346132 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:01.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.216+0000 7f8c47fff640 1 --2- 172.21.15.5:0/3682346132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c480cf710 0x7f8c4815c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.217+0000 7f8c477fe640 1 --2- 172.21.15.5:0/3682346132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c480d1670 0x7f8c4814f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.217+0000 7f8c47fff640 1 -- 172.21.15.5:0/3682346132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c480d1670 msgr2=0x7f8c4814f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.217+0000 7f0dd2d2e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0dbc003a10 0x7f0dcc0cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.217+0000 7f8c47fff640 1 --2- 172.21.15.5:0/3682346132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c480d1670 0x7f8c4814f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.218+0000 7f0dd2d2e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0dcc15c070 0x7f0dcc0ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.217+0000 7f8c47fff640 1 -- 172.21.15.5:0/3682346132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c4815d5d0 msgr2=0x7f8c4814fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.217+0000 7f8c47fff640 1 --2- 172.21.15.5:0/3682346132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c4815d5d0 0x7f8c4814fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.217+0000 7f8c47fff640 1 -- 172.21.15.5:0/3682346132 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c380094d0 con 0x7f8c480cf710 2023-11-24T06:16:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.217+0000 7f8c47fff640 1 --2- 172.21.15.5:0/3682346132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c480cf710 0x7f8c4815c340 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000c780 tx=0x7f8c4000cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.218+0000 7f0dd2d2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dcc0c70e0 0x7f0dcc0c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.218+0000 7f0dd2d2e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc0cf010 con 0x7f0dcc0c70e0 2023-11-24T06:16:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.218+0000 7f0dd2d2e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc0b8870 con 0x7f0dcc15c070 2023-11-24T06:16:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.218+0000 7f0dd2d2e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc15dd20 con 0x7f0dbc003a10 2023-11-24T06:16:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.216+0000 7fe583890640 1 -- start start 2023-11-24T06:16:01.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.218+0000 7f4c237fe640 1 --2- 172.21.15.5:0/4258316338 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c240abff0 0x7f4c240b25c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009e50 tx=0x7f4c1c02bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9afaae4513dc355 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.218+0000 7f0dd1d2c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0dbc003a10 0x7f0dcc0cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.218+0000 7f0dd1d2c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0dbc003a10 0x7f0dcc0cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:53534/0 (socket says 172.21.15.5:53534) 2023-11-24T06:16:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.218+0000 7f0dd1d2c640 1 -- 172.21.15.5:0/668667848 learned_addr learned my addr 172.21.15.5:0/668667848 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.218+0000 7f0dd252d640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dcc0c70e0 0x7f0dcc0c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.219+0000 7f0dd152b640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0dcc15c070 0x7f0dcc0ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.219+0000 7fe583890640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe57c15abe0 0x7fe57c0ae330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.219+0000 7fe583890640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe57c159c70 0x7fe57c15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.219+0000 7fe583890640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57c0ae9e0 con 0x7fe57c071fb0 2023-11-24T06:16:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.219+0000 7fe583890640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57c153f70 con 0x7fe57c15abe0 2023-11-24T06:16:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.219+0000 7fe583890640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57c06c550 con 0x7fe57c159c70 2023-11-24T06:16:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.219+0000 7f0dd1d2c640 1 -- 172.21.15.5:0/668667848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0dcc15c070 msgr2=0x7f0dcc0ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.219+0000 7f4c21ffb640 1 -- 172.21.15.5:0/4258316338 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c02ca80 con 0x7f4c240abff0 2023-11-24T06:16:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.219+0000 7f0dd1d2c640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0dcc15c070 0x7f0dcc0ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.219+0000 7f0dd1d2c640 1 -- 172.21.15.5:0/668667848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dcc0c70e0 msgr2=0x7f0dcc0c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.219+0000 7f4c21ffb640 1 -- 172.21.15.5:0/4258316338 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c02cbe0 con 0x7f4c240abff0 2023-11-24T06:16:01.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.219+0000 7f4c21ffb640 1 -- 172.21.15.5:0/4258316338 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c02ca80 con 0x7f4c240abff0 2023-11-24T06:16:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.219+0000 7fe580e2b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe57c15abe0 0x7fe57c0ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7fe580e2b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe57c15abe0 0x7fe57c0ae330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35964/0 (socket says 172.21.15.5:35964) 2023-11-24T06:16:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7fe580e2b640 1 -- 172.21.15.5:0/3737420561 learned_addr learned my addr 172.21.15.5:0/3737420561 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7f0dd1d2c640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dcc0c70e0 0x7f0dcc0c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7f0dd1d2c640 1 -- 172.21.15.5:0/668667848 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc15e000 con 0x7f0dbc003a10 2023-11-24T06:16:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7f0dd152b640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0dcc15c070 0x7f0dcc0ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:16:01.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7f0dd252d640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dcc0c70e0 0x7f0dcc0c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:16:01.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7fe581e2d640 1 --2- 172.21.15.5:0/3737420561 >> v2:172.21.15.6:3300/0 conn(0x7fe57c159c70 0x7fe57c15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.219+0000 7f8c44ff9640 1 -- 172.21.15.5:0/3682346132 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4000aa50 con 0x7f8c480cf710 2023-11-24T06:16:01.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7f8c44ff9640 1 -- 172.21.15.5:0/3682346132 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4000abd0 con 0x7f8c480cf710 2023-11-24T06:16:01.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7fe580e2b640 1 -- 172.21.15.5:0/3737420561 >> v2:172.21.15.6:3300/0 conn(0x7fe57c159c70 msgr2=0x7fe57c15a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:01.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7fe580e2b640 1 --2- 172.21.15.5:0/3737420561 >> v2:172.21.15.6:3300/0 conn(0x7fe57c159c70 0x7fe57c15a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7f8c44ff9640 1 -- 172.21.15.5:0/3682346132 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4000ce30 con 0x7f8c480cf710 2023-11-24T06:16:01.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7fe580e2b640 1 -- 172.21.15.5:0/3737420561 >> v1:172.21.15.6:6789/0 conn(0x7fe57c071fb0 legacy=0x7fe57c0adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:01.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7f8c4d49f640 1 -- 172.21.15.5:0/3682346132 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c480cb9b0 con 0x7f8c480cf710 2023-11-24T06:16:01.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7f8c4d49f640 1 -- 172.21.15.5:0/3682346132 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c480cbf70 con 0x7f8c480cf710 2023-11-24T06:16:01.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.220+0000 7f0dd1d2c640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0dbc003a10 0x7f0dcc0cc5b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400c290 tx=0x7f0dc4001930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.221+0000 7f4c29a1c640 1 -- 172.21.15.5:0/4258316338 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c240abff0 msgr2=0x7f4c240b25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.221+0000 7f4c29a1c640 1 --2- 172.21.15.5:0/4258316338 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c240abff0 0x7f4c240b25c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009e50 tx=0x7f4c1c02bee0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.221+0000 7fe580e2b640 1 -- 172.21.15.5:0/3737420561 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c06c830 con 0x7fe57c15abe0 2023-11-24T06:16:01.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.221+0000 7f0dca7fc640 1 -- 172.21.15.5:0/668667848 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4004da0 con 0x7f0dbc003a10 2023-11-24T06:16:01.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.222+0000 7f0dca7fc640 1 -- 172.21.15.5:0/668667848 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc4007070 con 0x7f0dbc003a10 2023-11-24T06:16:01.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.222+0000 7f0dca7fc640 1 -- 172.21.15.5:0/668667848 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40073e0 con 0x7f0dbc003a10 2023-11-24T06:16:01.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.222+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/668667848 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dcc15e2e0 con 0x7f0dbc003a10 2023-11-24T06:16:01.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.222+0000 7f8c4d49f640 1 -- 172.21.15.5:0/3682346132 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c480b8ee0 con 0x7f8c480cf710 2023-11-24T06:16:01.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.222+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/668667848 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dcc0c6bc0 con 0x7f0dbc003a10 2023-11-24T06:16:01.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.222+0000 7f4c29a1c640 1 -- 172.21.15.5:0/4258316338 shutdown_connections 2023-11-24T06:16:01.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.223+0000 7f4c29a1c640 1 --2- 172.21.15.5:0/4258316338 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c240abff0 0x7f4c240b25c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.223+0000 7f4c29a1c640 1 --2- 172.21.15.5:0/4258316338 >> v2:172.21.15.6:3300/0 conn(0x7f4c240ad720 0x7f4c240b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.223+0000 7f4c29a1c640 1 -- 172.21.15.5:0/4258316338 >> 172.21.15.5:0/4258316338 conn(0x7f4c240b0d80 msgr2=0x7f4c240b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.223+0000 7f4c29a1c640 1 -- 172.21.15.5:0/4258316338 shutdown_connections 2023-11-24T06:16:01.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.223+0000 7fe580e2b640 1 --2- 172.21.15.5:0/3737420561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57c15abe0 0x7fe57c0ae330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0ceab0 tx=0x7fe56c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba14a1c658c222f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.223+0000 7f4c29a1c640 1 -- 172.21.15.5:0/4258316338 wait complete. 2023-11-24T06:16:01.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.224+0000 7fe573fff640 1 -- 172.21.15.5:0/3737420561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c030860 con 0x7fe57c15abe0 2023-11-24T06:16:01.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.224+0000 7fe573fff640 1 -- 172.21.15.5:0/3737420561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c0309c0 con 0x7fe57c15abe0 2023-11-24T06:16:01.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.224+0000 7fe573fff640 1 -- 172.21.15.5:0/3737420561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c030d50 con 0x7fe57c15abe0 2023-11-24T06:16:01.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.224+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/668667848 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dcc0cfb10 con 0x7f0dbc003a10 2023-11-24T06:16:01.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.225+0000 7fe583890640 1 -- 172.21.15.5:0/3737420561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57c15abe0 msgr2=0x7fe57c0ae330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.225+0000 7fe583890640 1 --2- 172.21.15.5:0/3737420561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57c15abe0 0x7fe57c0ae330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0ceab0 tx=0x7fe56c02b9a0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.226+0000 7fe583890640 1 -- 172.21.15.5:0/3737420561 shutdown_connections 2023-11-24T06:16:01.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.226+0000 7fe583890640 1 --2- 172.21.15.5:0/3737420561 >> v2:172.21.15.6:3300/0 conn(0x7fe57c159c70 0x7fe57c15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.226+0000 7fe583890640 1 --2- 172.21.15.5:0/3737420561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57c15abe0 0x7fe57c0ae330 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.226+0000 7fe583890640 1 -- 172.21.15.5:0/3737420561 >> 172.21.15.5:0/3737420561 conn(0x7fe57c0709f0 msgr2=0x7fe57c0718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.226+0000 7fe583890640 1 -- 172.21.15.5:0/3737420561 shutdown_connections 2023-11-24T06:16:01.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.226+0000 7f4c29a1c640 1 Processor -- start 2023-11-24T06:16:01.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.226+0000 7fe583890640 1 -- 172.21.15.5:0/3737420561 wait complete. 2023-11-24T06:16:01.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.226+0000 7f4c29a1c640 1 -- start start 2023-11-24T06:16:01.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.226+0000 7f0dca7fc640 1 -- 172.21.15.5:0/668667848 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0dc4032070 con 0x7f0dbc003a10 2023-11-24T06:16:01.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.227+0000 7f8c44ff9640 1 -- 172.21.15.5:0/3682346132 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c40024430 con 0x7f8c480cf710 2023-11-24T06:16:01.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.229+0000 7fe583890640 1 Processor -- start 2023-11-24T06:16:01.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.228+0000 7f0dca7fc640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0db006aec0 0x7f0db006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.228+0000 7f0dca7fc640 1 -- 172.21.15.5:0/668667848 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0dc40aae00 con 0x7f0dbc003a10 2023-11-24T06:16:01.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.229+0000 7f4c29a1c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c2406d520 0x7f4c240cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.229+0000 7f4c29a1c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c240abff0 0x7f4c24099170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.229+0000 7f4c29a1c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240ad720 0x7f4c240996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.229+0000 7f4c29a1c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c24099f60 con 0x7f4c240ad720 2023-11-24T06:16:01.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.229+0000 7f4c29a1c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2409a520 con 0x7f4c2406d520 2023-11-24T06:16:01.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.229+0000 7f4c29a1c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2414e130 con 0x7f4c240abff0 2023-11-24T06:16:01.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.229+0000 7f8c44ff9640 1 --2- 172.21.15.5:0/3682346132 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c3406aec0 0x7f8c3406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.230+0000 7f4c22ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c2406d520 0x7f4c240cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.230+0000 7f4c22ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c2406d520 0x7f4c240cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:50242/0 (socket says 172.21.15.5:50242) 2023-11-24T06:16:01.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.230+0000 7f4c22ffd640 1 -- 172.21.15.5:0/3351025249 learned_addr learned my addr 172.21.15.5:0/3351025249 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:01.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.230+0000 7f4c237fe640 1 --2- 172.21.15.5:0/3351025249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240ad720 0x7f4c240996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.230+0000 7f8c44ff9640 1 -- 172.21.15.5:0/3682346132 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c40089a60 con 0x7f8c480cf710 2023-11-24T06:16:01.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.230+0000 7f8c477fe640 1 --2- 172.21.15.5:0/3682346132 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c3406aec0 0x7f8c3406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.231+0000 7f4c237fe640 1 -- 172.21.15.5:0/3351025249 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c240abff0 msgr2=0x7f4c24099170 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T06:16:01.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.231+0000 7f8c477fe640 1 --2- 172.21.15.5:0/3682346132 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c3406aec0 0x7f8c3406d380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8c38036000 tx=0x7f8c3803d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.229+0000 7fe583890640 1 -- start start 2023-11-24T06:16:01.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.231+0000 7f4c237fe640 1 --2- 172.21.15.5:0/3351025249 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c240abff0 0x7f4c24099170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.231+0000 7f4c237fe640 1 -- 172.21.15.5:0/3351025249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c2406d520 msgr2=0x7f4c240cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.231+0000 7f4c237fe640 1 --2- 172.21.15.5:0/3351025249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c2406d520 0x7f4c240cac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.231+0000 7f4c237fe640 1 -- 172.21.15.5:0/3351025249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c009b30 con 0x7f4c240ad720 2023-11-24T06:16:01.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.230+0000 7fe583890640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe57c071fb0 0x7fe57c09cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.230+0000 7fe583890640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe57c159c70 0x7fe57c0b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.230+0000 7fe583890640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57c15abe0 0x7fe57c136d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.231+0000 7fe583890640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c135c70 con 0x7fe57c15abe0 2023-11-24T06:16:01.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.231+0000 7fe583890640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c135f50 con 0x7fe57c071fb0 2023-11-24T06:16:01.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.231+0000 7fe583890640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c136230 con 0x7fe57c159c70 2023-11-24T06:16:01.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.231+0000 7fe581e2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57c15abe0 0x7fe57c136d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.231+0000 7fe58162c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe57c071fb0 0x7fe57c09cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.231+0000 7fe58162c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe57c071fb0 0x7fe57c09cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:50244/0 (socket says 172.21.15.5:50244) 2023-11-24T06:16:01.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.231+0000 7fe58162c640 1 -- 172.21.15.5:0/1897322389 learned_addr learned my addr 172.21.15.5:0/1897322389 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:01.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.231+0000 7fe580e2b640 1 --2- 172.21.15.5:0/1897322389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe57c159c70 0x7fe57c0b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.232+0000 7f0dd152b640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0db006aec0 0x7f0db006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.232+0000 7f4c22ffd640 1 --2- 172.21.15.5:0/3351025249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c2406d520 0x7f4c240cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:16:01.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.232+0000 7fe581e2d640 1 -- 172.21.15.5:0/1897322389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe57c159c70 msgr2=0x7fe57c0b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.232+0000 7fe581e2d640 1 --2- 172.21.15.5:0/1897322389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe57c159c70 0x7fe57c0b0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.232+0000 7fe581e2d640 1 -- 172.21.15.5:0/1897322389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe57c071fb0 msgr2=0x7fe57c09cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.232+0000 7fe581e2d640 1 --2- 172.21.15.5:0/1897322389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe57c071fb0 0x7fe57c09cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.232+0000 7fe581e2d640 1 -- 172.21.15.5:0/1897322389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c0094d0 con 0x7fe57c15abe0 2023-11-24T06:16:01.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.233+0000 7f4c237fe640 1 --2- 172.21.15.5:0/3351025249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240ad720 0x7f4c240996b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c032000 tx=0x7f4c1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.233+0000 7f0dd152b640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0db006aec0 0x7f0db006d380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0d0e40 tx=0x7f0db8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.233+0000 7fe581e2d640 1 --2- 172.21.15.5:0/1897322389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57c15abe0 0x7fe57c136d10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe57400efc0 tx=0x7fe57400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.233+0000 7f4c0bfff640 1 -- 172.21.15.5:0/3351025249 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c004250 con 0x7f4c240ad720 2023-11-24T06:16:01.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.233+0000 7f4c0bfff640 1 -- 172.21.15.5:0/3351025249 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1c0043b0 con 0x7f4c240ad720 2023-11-24T06:16:01.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.233+0000 7f4c0bfff640 1 -- 172.21.15.5:0/3351025249 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c0045a0 con 0x7f4c240ad720 2023-11-24T06:16:01.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.233+0000 7f4c29a1c640 1 -- 172.21.15.5:0/3351025249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2414e410 con 0x7f4c240ad720 2023-11-24T06:16:01.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.233+0000 7f4c29a1c640 1 -- 172.21.15.5:0/3351025249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2414de40 con 0x7f4c240ad720 2023-11-24T06:16:01.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.234+0000 7fe571ffb640 1 -- 172.21.15.5:0/1897322389 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574009c00 con 0x7fe57c15abe0 2023-11-24T06:16:01.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.234+0000 7fe571ffb640 1 -- 172.21.15.5:0/1897322389 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57400f040 con 0x7fe57c15abe0 2023-11-24T06:16:01.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.234+0000 7fe571ffb640 1 -- 172.21.15.5:0/1897322389 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5740146c0 con 0x7fe57c15abe0 2023-11-24T06:16:01.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.235+0000 7fe583890640 1 -- 172.21.15.5:0/1897322389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57c136510 con 0x7fe57c15abe0 2023-11-24T06:16:01.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.235+0000 7fe583890640 1 -- 172.21.15.5:0/1897322389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57c1358b0 con 0x7fe57c15abe0 2023-11-24T06:16:01.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.235+0000 7f4c29a1c640 1 -- 172.21.15.5:0/3351025249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2409a690 con 0x7f4c240ad720 2023-11-24T06:16:01.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.236+0000 7fe583890640 1 -- 172.21.15.5:0/1897322389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57c136a30 con 0x7fe57c15abe0 2023-11-24T06:16:01.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.238+0000 7f4c0bfff640 1 -- 172.21.15.5:0/3351025249 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c1c004700 con 0x7f4c240ad720 2023-11-24T06:16:01.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.238+0000 7f4c0bfff640 1 --2- 172.21.15.5:0/3351025249 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4c1006ae90 0x7f4c1006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.239+0000 7f4c0bfff640 1 -- 172.21.15.5:0/3351025249 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c1c0abe10 con 0x7f4c240ad720 2023-11-24T06:16:01.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.239+0000 7fe571ffb640 1 -- 172.21.15.5:0/1897322389 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe574014820 con 0x7fe57c15abe0 2023-11-24T06:16:01.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.240+0000 7f4c22ffd640 1 --2- 172.21.15.5:0/3351025249 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4c1006ae90 0x7f4c1006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.240+0000 7f8c44ff9640 1 -- 172.21.15.5:0/3682346132 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c40054c80 con 0x7f8c480cf710 2023-11-24T06:16:01.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.241+0000 7fe571ffb640 1 --2- 172.21.15.5:0/1897322389 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe56006af10 0x7fe56006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.241+0000 7fe571ffb640 1 -- 172.21.15.5:0/1897322389 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe57408a430 con 0x7fe57c15abe0 2023-11-24T06:16:01.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.241+0000 7fe58162c640 1 --2- 172.21.15.5:0/1897322389 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe56006af10 0x7fe56006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.242+0000 7f4c22ffd640 1 --2- 172.21.15.5:0/3351025249 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4c1006ae90 0x7f4c1006d350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4c2409d1f0 tx=0x7f4c18007ef0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.242+0000 7fe58162c640 1 --2- 172.21.15.5:0/1897322389 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe56006af10 0x7fe56006d3d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe578005020 tx=0x7fe578007ef0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.248+0000 7f4c0bfff640 1 -- 172.21.15.5:0/3351025249 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c1c077030 con 0x7f4c240ad720 2023-11-24T06:16:01.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.256+0000 7f0dca7fc640 1 -- 172.21.15.5:0/668667848 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0dc4076020 con 0x7f0dbc003a10 2023-11-24T06:16:01.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.258+0000 7fe571ffb640 1 -- 172.21.15.5:0/1897322389 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe57c136a30 con 0x7fe57c15abe0 2023-11-24T06:16:01.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.268+0000 7f96a562d640 1 Processor -- start 2023-11-24T06:16:01.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.268+0000 7f96a562d640 1 -- start start 2023-11-24T06:16:01.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.270+0000 7f96a562d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96a015d460 0x7f96a00b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.270+0000 7f96a562d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96a00d1300 0x7f96a00b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.270+0000 7f96a562d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f96a00d0230 0x7f96a00d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.270+0000 7f96a562d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a015d840 con 0x7f96a015d460 2023-11-24T06:16:01.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.270+0000 7f96a562d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96a00d0bc0 con 0x7f96a00d0230 2023-11-24T06:16:01.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.270+0000 7f96a562d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00d1790 con 0x7f96a00d1300 2023-11-24T06:16:01.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.271+0000 7f969effd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96a015d460 0x7f96a00b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.271+0000 7f969effd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96a015d460 0x7f96a00b62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35994/0 (socket says 172.21.15.5:35994) 2023-11-24T06:16:01.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.271+0000 7f969effd640 1 -- 172.21.15.5:0/3662242468 learned_addr learned my addr 172.21.15.5:0/3662242468 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:01.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.271+0000 7f969f7fe640 1 --2- 172.21.15.5:0/3662242468 >> v2:172.21.15.6:3300/0 conn(0x7f96a00d0230 0x7f96a00d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.272+0000 7f969effd640 1 -- 172.21.15.5:0/3662242468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96a00d1300 msgr2=0x7f96a00b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:01.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.272+0000 7f969effd640 1 --2- 172.21.15.5:0/3662242468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96a00d1300 0x7f96a00b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.272+0000 7f969effd640 1 -- 172.21.15.5:0/3662242468 >> v2:172.21.15.6:3300/0 conn(0x7f96a00d0230 msgr2=0x7f96a00d0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:01.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.272+0000 7f969effd640 1 --2- 172.21.15.5:0/3662242468 >> v2:172.21.15.6:3300/0 conn(0x7f96a00d0230 0x7f96a00d0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.272+0000 7f969effd640 1 -- 172.21.15.5:0/3662242468 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a00b69a0 con 0x7f96a015d460 2023-11-24T06:16:01.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.274+0000 7f969effd640 1 --2- 172.21.15.5:0/3662242468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a015d460 0x7f96a00b62a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f96a00b51c0 tx=0x7f96980346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72343c48bdf4857e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.275+0000 7f969dffb640 1 -- 172.21.15.5:0/3662242468 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969803b600 con 0x7f96a015d460 2023-11-24T06:16:01.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.275+0000 7f969dffb640 1 -- 172.21.15.5:0/3662242468 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9698037050 con 0x7f96a015d460 2023-11-24T06:16:01.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.275+0000 7f969dffb640 1 -- 172.21.15.5:0/3662242468 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969803b980 con 0x7f96a015d460 2023-11-24T06:16:01.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.275+0000 7f2e8821c640 1 Processor -- start 2023-11-24T06:16:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.276+0000 7f96a562d640 1 -- 172.21.15.5:0/3662242468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a015d460 msgr2=0x7f96a00b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.276+0000 7f96a562d640 1 --2- 172.21.15.5:0/3662242468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a015d460 0x7f96a00b62a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f96a00b51c0 tx=0x7f96980346f0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.277+0000 7f96a562d640 1 -- 172.21.15.5:0/3662242468 shutdown_connections 2023-11-24T06:16:01.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.277+0000 7f96a562d640 1 --2- 172.21.15.5:0/3662242468 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96a00d1300 0x7f96a00b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.277+0000 7f96a562d640 1 --2- 172.21.15.5:0/3662242468 >> v2:172.21.15.6:3300/0 conn(0x7f96a00d0230 0x7f96a00d0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.277+0000 7f96a562d640 1 --2- 172.21.15.5:0/3662242468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a015d460 0x7f96a00b62a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.277+0000 7f96a562d640 1 -- 172.21.15.5:0/3662242468 >> 172.21.15.5:0/3662242468 conn(0x7f96a0071f90 msgr2=0x7f96a00723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.276+0000 7f2e8821c640 1 -- start start 2023-11-24T06:16:01.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.277+0000 7f96a562d640 1 -- 172.21.15.5:0/3662242468 shutdown_connections 2023-11-24T06:16:01.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.278+0000 7f2e8821c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2e800ae330 0x7f2e80164050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.278+0000 7f2e8821c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e800ae780 con 0x7f2e80159d00 2023-11-24T06:16:01.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.278+0000 7f2e8821c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e800ae9e0 con 0x7f2e8015abe0 2023-11-24T06:16:01.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.278+0000 7f2e8821c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e80153f70 con 0x7f2e800ae330 2023-11-24T06:16:01.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.279+0000 7f2e85fb8640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2e80159d00 0x7f2e8015a0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56602/0 (socket says 172.21.15.5:56602) 2023-11-24T06:16:01.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.279+0000 7f2e85fb8640 1 -- 172.21.15.5:0/160173056 learned_addr learned my addr 172.21.15.5:0/160173056 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:01.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.279+0000 7f2e867b9640 1 --2- 172.21.15.5:0/160173056 >> v2:172.21.15.6:3300/0 conn(0x7f2e800ae330 0x7f2e80164050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.280+0000 7f2e84fb6640 1 -- 172.21.15.5:0/160173056 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741318558 0 0) 0x7f2e800ae780 con 0x7f2e80159d00 2023-11-24T06:16:01.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.280+0000 7f2e84fb6640 1 -- 172.21.15.5:0/160173056 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e6c004630 con 0x7f2e80159d00 2023-11-24T06:16:01.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.281+0000 7f2e867b9640 1 -- 172.21.15.5:0/160173056 >> v1:172.21.15.6:6789/0 conn(0x7f2e8015abe0 legacy=0x7f2e800adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.281+0000 7f2e84fb6640 1 -- 172.21.15.5:0/160173056 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3844508403 0 0) 0x7f2e6c004630 con 0x7f2e80159d00 2023-11-24T06:16:01.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.277+0000 7f96a562d640 1 -- 172.21.15.5:0/3662242468 wait complete. 2023-11-24T06:16:01.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.280+0000 7f96a562d640 1 Processor -- start 2023-11-24T06:16:01.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.281+0000 7f96a562d640 1 -- start start 2023-11-24T06:16:01.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.282+0000 7f96a562d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96a00d0230 0x7f96a0155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.282+0000 7f96a562d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96a00d1300 0x7f96a0149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.282+0000 7f96a562d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a015d460 0x7f96a0086040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.281+0000 7f2e867b9640 1 -- 172.21.15.5:0/160173056 >> v1:172.21.15.5:6789/0 conn(0x7f2e80159d00 legacy=0x7f2e8015a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.281+0000 7f2e867b9640 1 -- 172.21.15.5:0/160173056 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e8006c550 con 0x7f2e800ae330 2023-11-24T06:16:01.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.283+0000 7f96a562d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a0156620 con 0x7f96a015d460 2023-11-24T06:16:01.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.283+0000 7f96a562d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00c6e90 con 0x7f96a00d1300 2023-11-24T06:16:01.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.283+0000 7f96a562d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96a0155aa0 con 0x7f96a00d0230 2023-11-24T06:16:01.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.283+0000 7f969effd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96a00d0230 0x7f96a0155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.283+0000 7f969effd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96a00d0230 0x7f96a0155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:53596/0 (socket says 172.21.15.5:53596) 2023-11-24T06:16:01.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.283+0000 7f969effd640 1 -- 172.21.15.5:0/3651229897 learned_addr learned my addr 172.21.15.5:0/3651229897 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:01.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.283+0000 7f969f7fe640 1 --2- 172.21.15.5:0/3651229897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a015d460 0x7f96a0086040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.283+0000 7f969e7fc640 1 --2- 172.21.15.5:0/3651229897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96a00d1300 0x7f96a0149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.284+0000 7f2e867b9640 1 --2- 172.21.15.5:0/160173056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e800ae330 0x7f2e80164050 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2e700097a0 tx=0x7f2e7002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59c148a5da0d54a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.284+0000 7f969effd640 1 -- 172.21.15.5:0/3651229897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96a00d1300 msgr2=0x7f96a0149300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.284+0000 7f969effd640 1 --2- 172.21.15.5:0/3651229897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96a00d1300 0x7f96a0149300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.284+0000 7f969effd640 1 -- 172.21.15.5:0/3651229897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a015d460 msgr2=0x7f96a0086040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.284+0000 7f969effd640 1 --2- 172.21.15.5:0/3651229897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a015d460 0x7f96a0086040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.285+0000 7f2e84fb6640 1 -- 172.21.15.5:0/160173056 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7002f050 con 0x7f2e800ae330 2023-11-24T06:16:01.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.285+0000 7f2e84fb6640 1 -- 172.21.15.5:0/160173056 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e70031500 con 0x7f2e800ae330 2023-11-24T06:16:01.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.285+0000 7f2e84fb6640 1 -- 172.21.15.5:0/160173056 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c004cd0 con 0x7f2e800ae330 2023-11-24T06:16:01.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.284+0000 7f969effd640 1 -- 172.21.15.5:0/3651229897 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9698009b30 con 0x7f96a00d0230 2023-11-24T06:16:01.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.285+0000 7f969effd640 1 --2- 172.21.15.5:0/3651229897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96a00d0230 0x7f96a0155460 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9698000f80 tx=0x7f9698004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.286+0000 7f2e8821c640 1 -- 172.21.15.5:0/160173056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e800ae330 msgr2=0x7f2e80164050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.286+0000 7f2e8821c640 1 --2- 172.21.15.5:0/160173056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e800ae330 0x7f2e80164050 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2e700097a0 tx=0x7f2e7002b820 comp rx=0 tx=0).stop 2023-11-24T06:16:01.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.286+0000 7f2e8821c640 1 -- 172.21.15.5:0/160173056 shutdown_connections 2023-11-24T06:16:01.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.286+0000 7f2e8821c640 1 --2- 172.21.15.5:0/160173056 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e800ae330 0x7f2e80164050 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.286+0000 7f2e8821c640 1 -- 172.21.15.5:0/160173056 >> 172.21.15.5:0/160173056 conn(0x7f2e800709f0 msgr2=0x7f2e800718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.286+0000 7f2e8821c640 1 -- 172.21.15.5:0/160173056 shutdown_connections 2023-11-24T06:16:01.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.286+0000 7f2e8821c640 1 -- 172.21.15.5:0/160173056 wait complete. 2023-11-24T06:16:01.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.286+0000 7f96877fe640 1 -- 172.21.15.5:0/3651229897 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9698047860 con 0x7f96a00d0230 2023-11-24T06:16:01.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.286+0000 7f96877fe640 1 -- 172.21.15.5:0/3651229897 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96980479c0 con 0x7f96a00d0230 2023-11-24T06:16:01.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.287+0000 7f96877fe640 1 -- 172.21.15.5:0/3651229897 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9698047d70 con 0x7f96a00d0230 2023-11-24T06:16:01.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.287+0000 7f96a562d640 1 -- 172.21.15.5:0/3651229897 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a0155d80 con 0x7f96a00d0230 2023-11-24T06:16:01.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.288+0000 7f96a562d640 1 -- 172.21.15.5:0/3651229897 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a0156340 con 0x7f96a00d0230 2023-11-24T06:16:01.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.289+0000 7f2e8821c640 1 Processor -- start 2023-11-24T06:16:01.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.289+0000 7f2e8821c640 1 -- start start 2023-11-24T06:16:01.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.290+0000 7f96857fa640 1 -- 172.21.15.5:0/3651229897 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9678005250 con 0x7f96a00d0230 2023-11-24T06:16:01.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.291+0000 7f2e8821c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e800ae330 0x7f2e800b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.291+0000 7f2e8821c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e80159d00 0x7f2e800b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.291+0000 7f2e8821c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e8015abe0 0x7f2e800b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.291+0000 7f2e8821c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800b3a10 con 0x7f2e800ae330 2023-11-24T06:16:01.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.291+0000 7f2e8821c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800b2430 con 0x7f2e8015abe0 2023-11-24T06:16:01.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.291+0000 7f2e8821c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800b2710 con 0x7f2e80159d00 2023-11-24T06:16:01.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.292+0000 7f2e85fb8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e800ae330 0x7f2e800b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.292+0000 7f2e85fb8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e800ae330 0x7f2e800b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35998/0 (socket says 172.21.15.5:35998) 2023-11-24T06:16:01.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.292+0000 7f2e85fb8640 1 -- 172.21.15.5:0/112122632 learned_addr learned my addr 172.21.15.5:0/112122632 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:01.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.292+0000 7f2e867b9640 1 --2- 172.21.15.5:0/112122632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e8015abe0 0x7f2e800b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.292+0000 7f2e857b7640 1 --2- 172.21.15.5:0/112122632 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e80159d00 0x7f2e800b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.292+0000 7f2e85fb8640 1 -- 172.21.15.5:0/112122632 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e80159d00 msgr2=0x7f2e800b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.292+0000 7f2e85fb8640 1 --2- 172.21.15.5:0/112122632 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e80159d00 0x7f2e800b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.292+0000 7f2e85fb8640 1 -- 172.21.15.5:0/112122632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e8015abe0 msgr2=0x7f2e800b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.292+0000 7f2e85fb8640 1 --2- 172.21.15.5:0/112122632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e8015abe0 0x7f2e800b1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.292+0000 7f2e85fb8640 1 -- 172.21.15.5:0/112122632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e70009400 con 0x7f2e800ae330 2023-11-24T06:16:01.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.293+0000 7f2e85fb8640 1 --2- 172.21.15.5:0/112122632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e800ae330 0x7f2e800b0520 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00d930 tx=0x7f2e7c00ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.294+0000 7f96877fe640 1 -- 172.21.15.5:0/3651229897 <== mon.2 v2:172.21.15.6:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9698045400 con 0x7f96a00d0230 2023-11-24T06:16:01.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.294+0000 7f2e767fc640 1 -- 172.21.15.5:0/112122632 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c008b70 con 0x7f2e800ae330 2023-11-24T06:16:01.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.294+0000 7f2e767fc640 1 -- 172.21.15.5:0/112122632 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e7c008cd0 con 0x7f2e800ae330 2023-11-24T06:16:01.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.294+0000 7f2e767fc640 1 -- 172.21.15.5:0/112122632 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c010650 con 0x7f2e800ae330 2023-11-24T06:16:01.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.295+0000 7f2e8821c640 1 -- 172.21.15.5:0/112122632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e800b29f0 con 0x7f2e800ae330 2023-11-24T06:16:01.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.295+0000 7f2e8821c640 1 -- 172.21.15.5:0/112122632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e800b2fb0 con 0x7f2e800ae330 2023-11-24T06:16:01.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.295+0000 7f96877fe640 1 --2- 172.21.15.5:0/3651229897 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f968c06ae90 0x7f968c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.295+0000 7f96877fe640 1 -- 172.21.15.5:0/3651229897 <== mon.2 v2:172.21.15.6:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96980b43a0 con 0x7f96a00d0230 2023-11-24T06:16:01.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.296+0000 7f969e7fc640 1 --2- 172.21.15.5:0/3651229897 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f968c06ae90 0x7f968c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.296+0000 7f2e8821c640 1 -- 172.21.15.5:0/112122632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e800b31a0 con 0x7f2e800ae330 2023-11-24T06:16:01.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.297+0000 7f969e7fc640 1 --2- 172.21.15.5:0/3651229897 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f968c06ae90 0x7f968c06d350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f96a0148ff0 tx=0x7f9690009110 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.299+0000 7f2e767fc640 1 -- 172.21.15.5:0/112122632 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e7c0107b0 con 0x7f2e800ae330 2023-11-24T06:16:01.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.300+0000 7f2e767fc640 1 --2- 172.21.15.5:0/112122632 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2e6406ae90 0x7f2e6406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.300+0000 7ffa0da31640 1 Processor -- start 2023-11-24T06:16:01.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.301+0000 7ffa0da31640 1 -- start start 2023-11-24T06:16:01.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.304+0000 7f2e767fc640 1 -- 172.21.15.5:0/112122632 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e7c089d10 con 0x7f2e800ae330 2023-11-24T06:16:01.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.303+0000 7ffa0da31640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa0806eb00 0x7ffa0806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.303+0000 7ffa0da31640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ffa080cfa00 0x7ffa080cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.303+0000 7ffa0da31640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa080ce230 0x7ffa080d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.303+0000 7ffa0da31640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0815c3e0 con 0x7ffa0806eb00 2023-11-24T06:16:01.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.303+0000 7ffa0da31640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0815b840 con 0x7ffa080cfa00 2023-11-24T06:16:01.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.303+0000 7ffa0da31640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0806f420 con 0x7ffa080ce230 2023-11-24T06:16:01.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.304+0000 7ffa06ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa0806eb00 0x7ffa0806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.304+0000 7ffa06ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffa0806eb00 0x7ffa0806eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36006/0 (socket says 172.21.15.5:36006) 2023-11-24T06:16:01.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.304+0000 7ffa06ffd640 1 -- 172.21.15.5:0/1000521498 learned_addr learned my addr 172.21.15.5:0/1000521498 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:01.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.304+0000 7ffa077fe640 1 --2- 172.21.15.5:0/1000521498 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa080ce230 0x7ffa080d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.305+0000 7f2e857b7640 1 --2- 172.21.15.5:0/112122632 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2e6406ae90 0x7f2e6406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.306+0000 7f2e857b7640 1 --2- 172.21.15.5:0/112122632 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2e6406ae90 0x7f2e6406d350 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2e80155c60 tx=0x7f2e78009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.305+0000 7ffa077fe640 1 -- 172.21.15.5:0/1000521498 >> v2:172.21.15.6:3300/0 conn(0x7ffa080cfa00 msgr2=0x7ffa080cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:01.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.305+0000 7ffa077fe640 1 --2- 172.21.15.5:0/1000521498 >> v2:172.21.15.6:3300/0 conn(0x7ffa080cfa00 0x7ffa080cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.305+0000 7ffa077fe640 1 -- 172.21.15.5:0/1000521498 >> v2:172.21.15.5:3300/0 conn(0x7ffa0806eb00 msgr2=0x7ffa0806eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:01.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.305+0000 7ffa077fe640 1 --2- 172.21.15.5:0/1000521498 >> v2:172.21.15.5:3300/0 conn(0x7ffa0806eb00 0x7ffa0806eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.305+0000 7ffa077fe640 1 -- 172.21.15.5:0/1000521498 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0815bab0 con 0x7ffa080ce230 2023-11-24T06:16:01.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.307+0000 7ffa077fe640 1 --2- 172.21.15.5:0/1000521498 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa080ce230 0x7ffa080d0fe0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffa08141910 tx=0x7ffa0002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9ffb1af3bd94946 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.308+0000 7ffa067fc640 1 -- 172.21.15.5:0/1000521498 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0002e070 con 0x7ffa080ce230 2023-11-24T06:16:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.309+0000 7ffa067fc640 1 -- 172.21.15.5:0/1000521498 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa000315a0 con 0x7ffa080ce230 2023-11-24T06:16:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.309+0000 7ffa067fc640 1 -- 172.21.15.5:0/1000521498 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa00031930 con 0x7ffa080ce230 2023-11-24T06:16:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.309+0000 7f2e767fc640 1 -- 172.21.15.5:0/112122632 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e7c08f020 con 0x7f2e800ae330 2023-11-24T06:16:01.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.309+0000 7f96877fe640 1 -- 172.21.15.5:0/3651229897 <== mon.2 v2:172.21.15.6:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f969807f4c0 con 0x7f96a00d0230 2023-11-24T06:16:01.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.310+0000 7ffa0da31640 1 -- 172.21.15.5:0/1000521498 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa080ce230 msgr2=0x7ffa080d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.310+0000 7ffa0da31640 1 --2- 172.21.15.5:0/1000521498 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa080ce230 0x7ffa080d0fe0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ffa08141910 tx=0x7ffa0002b840 comp rx=0 tx=0).stop 2023-11-24T06:16:01.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.310+0000 7ffa0da31640 1 -- 172.21.15.5:0/1000521498 shutdown_connections 2023-11-24T06:16:01.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.310+0000 7ffa0da31640 1 --2- 172.21.15.5:0/1000521498 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa080ce230 0x7ffa080d0fe0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.310+0000 7ffa0da31640 1 --2- 172.21.15.5:0/1000521498 >> v2:172.21.15.6:3300/0 conn(0x7ffa080cfa00 0x7ffa080cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.310+0000 7ffa0da31640 1 --2- 172.21.15.5:0/1000521498 >> v2:172.21.15.5:3300/0 conn(0x7ffa0806eb00 0x7ffa0806eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.310+0000 7ffa0da31640 1 -- 172.21.15.5:0/1000521498 >> 172.21.15.5:0/1000521498 conn(0x7ffa080663d0 msgr2=0x7ffa080667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.310+0000 7ffa0da31640 1 -- 172.21.15.5:0/1000521498 shutdown_connections 2023-11-24T06:16:01.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.310+0000 7ffa0da31640 1 -- 172.21.15.5:0/1000521498 wait complete. 2023-11-24T06:16:01.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.313+0000 7ffa0da31640 1 Processor -- start 2023-11-24T06:16:01.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.314+0000 7ffa0da31640 1 -- start start 2023-11-24T06:16:01.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.316+0000 7ffa0da31640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa0806eb00 0x7ffa081f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.316+0000 7ffa0da31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa080ce230 0x7ffa081551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.316+0000 7ffa0da31640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa080cfa00 0x7ffa08152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.316+0000 7ffa0da31640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa08158f70 con 0x7ffa080ce230 2023-11-24T06:16:01.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.316+0000 7ffa0da31640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa08130ce0 con 0x7ffa0806eb00 2023-11-24T06:16:01.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.317+0000 7ffa0da31640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa081582f0 con 0x7ffa080cfa00 2023-11-24T06:16:01.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.317+0000 7ffa06ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa0806eb00 0x7ffa081f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.317+0000 7ffa06ffd640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa0806eb00 0x7ffa081f0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:50296/0 (socket says 172.21.15.5:50296) 2023-11-24T06:16:01.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.317+0000 7ffa06ffd640 1 -- 172.21.15.5:0/2750653367 learned_addr learned my addr 172.21.15.5:0/2750653367 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:01.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.317+0000 7ffa077fe640 1 --2- 172.21.15.5:0/2750653367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa080cfa00 0x7ffa08152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.317+0000 7ffa06ffd640 1 -- 172.21.15.5:0/2750653367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa080cfa00 msgr2=0x7ffa08152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.317+0000 7ffa06ffd640 1 --2- 172.21.15.5:0/2750653367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa080cfa00 0x7ffa08152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.317+0000 7ffa06ffd640 1 -- 172.21.15.5:0/2750653367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa080ce230 msgr2=0x7ffa081551a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T06:16:01.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.320+0000 7ff9fe7fc640 1 --2- 172.21.15.5:0/2750653367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa080ce230 0x7ffa081551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.320+0000 7ffa06ffd640 1 --2- 172.21.15.5:0/2750653367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa080ce230 0x7ffa081551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.320+0000 7ffa06ffd640 1 -- 172.21.15.5:0/2750653367 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa00009400 con 0x7ffa0806eb00 2023-11-24T06:16:01.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.322+0000 7ffa06ffd640 1 --2- 172.21.15.5:0/2750653367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa0806eb00 0x7ffa081f0f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800efc0 tx=0x7ff9f800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.322+0000 7ffa04ff9640 1 -- 172.21.15.5:0/2750653367 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8009b60 con 0x7ffa0806eb00 2023-11-24T06:16:01.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.322+0000 7ffa04ff9640 1 -- 172.21.15.5:0/2750653367 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f800f040 con 0x7ffa0806eb00 2023-11-24T06:16:01.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.322+0000 7ffa04ff9640 1 -- 172.21.15.5:0/2750653367 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80092e0 con 0x7ffa0806eb00 2023-11-24T06:16:01.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.324+0000 7ffa0da31640 1 -- 172.21.15.5:0/2750653367 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa081585d0 con 0x7ffa0806eb00 2023-11-24T06:16:01.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.324+0000 7ffa0da31640 1 -- 172.21.15.5:0/2750653367 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa08158b90 con 0x7ffa0806eb00 2023-11-24T06:16:01.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.326+0000 7ffa0da31640 1 -- 172.21.15.5:0/2750653367 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa08152850 con 0x7ffa0806eb00 2023-11-24T06:16:01.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.329+0000 7ffa04ff9640 1 -- 172.21.15.5:0/2750653367 <== mon.1 v2:172.21.15.6:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff9f8014480 con 0x7ffa0806eb00 2023-11-24T06:16:01.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.330+0000 7ffa04ff9640 1 --2- 172.21.15.5:0/2750653367 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff9e806ae40 0x7ff9e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:01.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.331+0000 7ffa04ff9640 1 -- 172.21.15.5:0/2750653367 <== mon.1 v2:172.21.15.6:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9f8089d70 con 0x7ffa0806eb00 2023-11-24T06:16:01.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.332+0000 7ff9fe7fc640 1 --2- 172.21.15.5:0/2750653367 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff9e806ae40 0x7ff9e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:01.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.333+0000 7ff9fe7fc640 1 --2- 172.21.15.5:0/2750653367 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff9e806ae40 0x7ff9e806d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffa081592d0 tx=0x7ff9f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:01.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.344+0000 7ffa04ff9640 1 -- 172.21.15.5:0/2750653367 <== mon.1 v2:172.21.15.6:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff9f8054cf0 con 0x7ffa0806eb00 2023-11-24T06:16:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.609+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/556751001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7aac052740 con 0x7f7aac105390 2023-11-24T06:16:01.613 INFO:teuthology.orchestra.run.smithi005.stdout:60129542150 2023-11-24T06:16:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.610+0000 7f7a9b7fe640 1 -- 172.21.15.5:0/556751001 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7aa8057de0 con 0x7f7aac105390 2023-11-24T06:16:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.614+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/556751001 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7aa006af40 msgr2=0x7f7aa006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.614+0000 7f7ab8a0d640 1 --2- 172.21.15.5:0/556751001 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7aa006af40 0x7f7aa006d400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40753a0 tx=0x7f7ab4099000 comp rx=0 tx=0).stop 2023-11-24T06:16:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.614+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/556751001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aac105390 msgr2=0x7f7aac0485c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.614+0000 7f7ab8a0d640 1 --2- 172.21.15.5:0/556751001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aac105390 0x7f7aac0485c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800eb60 tx=0x7f7aa800c520 comp rx=0 tx=0).stop 2023-11-24T06:16:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.615+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/556751001 shutdown_connections 2023-11-24T06:16:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.615+0000 7f7ab8a0d640 1 --2- 172.21.15.5:0/556751001 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7aa006af40 0x7f7aa006d400 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.615+0000 7f7ab8a0d640 1 --2- 172.21.15.5:0/556751001 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7aac100ee0 0x7f7aac036220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.615+0000 7f7ab8a0d640 1 --2- 172.21.15.5:0/556751001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aac105390 0x7f7aac0485c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.615+0000 7f7ab8a0d640 1 --2- 172.21.15.5:0/556751001 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7aac0544f0 0x7f7aac0f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.615+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/556751001 >> 172.21.15.5:0/556751001 conn(0x7f7aac009f80 msgr2=0x7f7aac103390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.616+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/556751001 shutdown_connections 2023-11-24T06:16:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.616+0000 7f7ab8a0d640 1 -- 172.21.15.5:0/556751001 wait complete. 2023-11-24T06:16:01.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.0 2023-11-24T06:16:01.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.643+0000 7f8c4d49f640 1 -- 172.21.15.5:0/3682346132 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8c4804e840 con 0x7f8c480cf710 2023-11-24T06:16:01.647 DEBUG:teuthology.parallel:result is None 2023-11-24T06:16:01.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.644+0000 7f8c44ff9640 1 -- 172.21.15.5:0/3682346132 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8c400582a0 con 0x7f8c480cf710 2023-11-24T06:16:01.647 INFO:teuthology.orchestra.run.smithi005.stdout:60129542150 2023-11-24T06:16:01.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.647+0000 7f8c4d49f640 1 -- 172.21.15.5:0/3682346132 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c3406aec0 msgr2=0x7f8c3406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.647+0000 7f8c4d49f640 1 --2- 172.21.15.5:0/3682346132 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c3406aec0 0x7f8c3406d380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8c38036000 tx=0x7f8c3803d040 comp rx=0 tx=0).stop 2023-11-24T06:16:01.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.648+0000 7f8c4d49f640 1 -- 172.21.15.5:0/3682346132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c480cf710 msgr2=0x7f8c4815c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.648+0000 7f8c4d49f640 1 --2- 172.21.15.5:0/3682346132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c480cf710 0x7f8c4815c340 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8c4000c780 tx=0x7f8c4000cc40 comp rx=0 tx=0).stop 2023-11-24T06:16:01.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.648+0000 7f8c4d49f640 1 -- 172.21.15.5:0/3682346132 shutdown_connections 2023-11-24T06:16:01.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.648+0000 7f8c4d49f640 1 --2- 172.21.15.5:0/3682346132 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c3406aec0 0x7f8c3406d380 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.648+0000 7f8c4d49f640 1 --2- 172.21.15.5:0/3682346132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c4815d5d0 0x7f8c4814fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.648+0000 7f8c4d49f640 1 --2- 172.21.15.5:0/3682346132 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c480d1670 0x7f8c4814f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.648+0000 7f8c4d49f640 1 --2- 172.21.15.5:0/3682346132 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c480cf710 0x7f8c4815c340 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.648+0000 7f8c4d49f640 1 -- 172.21.15.5:0/3682346132 >> 172.21.15.5:0/3682346132 conn(0x7f8c4809c3d0 msgr2=0x7f8c480ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.649+0000 7f8c4d49f640 1 -- 172.21.15.5:0/3682346132 shutdown_connections 2023-11-24T06:16:01.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.649+0000 7f8c4d49f640 1 -- 172.21.15.5:0/3682346132 wait complete. 2023-11-24T06:16:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.660+0000 7fe583890640 1 -- 172.21.15.5:0/1897322389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe57c136a30 con 0x7fe57c15abe0 2023-11-24T06:16:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.660+0000 7fe571ffb640 1 -- 172.21.15.5:0/1897322389 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe57c136a30 con 0x7fe57c15abe0 2023-11-24T06:16:01.664 INFO:teuthology.orchestra.run.smithi005.stdout:60129542150 2023-11-24T06:16:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.663+0000 7fe583890640 1 -- 172.21.15.5:0/1897322389 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe56006af10 msgr2=0x7fe56006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.663+0000 7fe583890640 1 --2- 172.21.15.5:0/1897322389 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe56006af10 0x7fe56006d3d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe578005020 tx=0x7fe578007ef0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.663+0000 7fe583890640 1 -- 172.21.15.5:0/1897322389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57c15abe0 msgr2=0x7fe57c136d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.663+0000 7fe583890640 1 --2- 172.21.15.5:0/1897322389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57c15abe0 0x7fe57c136d10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe57400efc0 tx=0x7fe57400c490 comp rx=0 tx=0).stop 2023-11-24T06:16:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.663+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/668667848 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0dcc15de60 con 0x7f0dbc003a10 2023-11-24T06:16:01.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.663+0000 7fe583890640 1 -- 172.21.15.5:0/1897322389 shutdown_connections 2023-11-24T06:16:01.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.663+0000 7fe583890640 1 --2- 172.21.15.5:0/1897322389 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe56006af10 0x7fe56006d3d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.663+0000 7fe583890640 1 --2- 172.21.15.5:0/1897322389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe57c159c70 0x7fe57c0b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.663+0000 7fe583890640 1 --2- 172.21.15.5:0/1897322389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe57c15abe0 0x7fe57c136d10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.663+0000 7fe583890640 1 --2- 172.21.15.5:0/1897322389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe57c071fb0 0x7fe57c09cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.663+0000 7fe583890640 1 -- 172.21.15.5:0/1897322389 >> 172.21.15.5:0/1897322389 conn(0x7fe57c0709f0 msgr2=0x7fe57c158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.663+0000 7fe583890640 1 -- 172.21.15.5:0/1897322389 shutdown_connections 2023-11-24T06:16:01.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.664+0000 7fe583890640 1 -- 172.21.15.5:0/1897322389 wait complete. 2023-11-24T06:16:01.668 INFO:teuthology.orchestra.run.smithi005.stdout:60129542151 2023-11-24T06:16:01.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.664+0000 7f0dca7fc640 1 -- 172.21.15.5:0/668667848 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0dcc15de60 con 0x7f0dbc003a10 2023-11-24T06:16:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.668+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/668667848 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0db006aec0 msgr2=0x7f0db006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.668+0000 7f0dd2d2e640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0db006aec0 0x7f0db006d380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0d0e40 tx=0x7f0db8009040 comp rx=0 tx=0).stop 2023-11-24T06:16:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.668+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/668667848 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0dbc003a10 msgr2=0x7f0dcc0cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.668+0000 7f0dd2d2e640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0dbc003a10 0x7f0dcc0cc5b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400c290 tx=0x7f0dc4001930 comp rx=0 tx=0).stop 2023-11-24T06:16:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.668+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/668667848 shutdown_connections 2023-11-24T06:16:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.668+0000 7f0dd2d2e640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0db006aec0 0x7f0db006d380 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.668+0000 7f0dd2d2e640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dcc0c70e0 0x7f0dcc0c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.668+0000 7f0dd2d2e640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0dcc15c070 0x7f0dcc0ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.668+0000 7f0dd2d2e640 1 --2- 172.21.15.5:0/668667848 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0dbc003a10 0x7f0dcc0cc5b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.668+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/668667848 >> 172.21.15.5:0/668667848 conn(0x7f0dcc09c3d0 msgr2=0x7f0dcc0ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.669+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/668667848 shutdown_connections 2023-11-24T06:16:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.669+0000 7f0dd2d2e640 1 -- 172.21.15.5:0/668667848 wait complete. 2023-11-24T06:16:01.679 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.1 2023-11-24T06:16:01.679 DEBUG:teuthology.parallel:result is None 2023-11-24T06:16:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.677+0000 7f4c29a1c640 1 -- 172.21.15.5:0/3351025249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4c24074770 con 0x7f4c240ad720 2023-11-24T06:16:01.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.678+0000 7f4c0bfff640 1 -- 172.21.15.5:0/3351025249 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4c1c07a650 con 0x7f4c240ad720 2023-11-24T06:16:01.681 INFO:teuthology.orchestra.run.smithi005.stdout:60129542150 2023-11-24T06:16:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.681+0000 7f4c29a1c640 1 -- 172.21.15.5:0/3351025249 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4c1006ae90 msgr2=0x7f4c1006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.681+0000 7f4c29a1c640 1 --2- 172.21.15.5:0/3351025249 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4c1006ae90 0x7f4c1006d350 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4c2409d1f0 tx=0x7f4c18007ef0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.681+0000 7f4c29a1c640 1 -- 172.21.15.5:0/3351025249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240ad720 msgr2=0x7f4c240996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.681+0000 7f4c29a1c640 1 --2- 172.21.15.5:0/3351025249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240ad720 0x7f4c240996b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c032000 tx=0x7f4c1c004a40 comp rx=0 tx=0).stop 2023-11-24T06:16:01.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.682+0000 7f4c29a1c640 1 -- 172.21.15.5:0/3351025249 shutdown_connections 2023-11-24T06:16:01.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.682+0000 7f4c29a1c640 1 --2- 172.21.15.5:0/3351025249 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4c1006ae90 0x7f4c1006d350 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.682+0000 7f4c29a1c640 1 --2- 172.21.15.5:0/3351025249 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c240abff0 0x7f4c24099170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.682+0000 7f4c29a1c640 1 --2- 172.21.15.5:0/3351025249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c240ad720 0x7f4c240996b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.682+0000 7f4c29a1c640 1 --2- 172.21.15.5:0/3351025249 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c2406d520 0x7f4c240cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.682+0000 7f4c29a1c640 1 -- 172.21.15.5:0/3351025249 >> 172.21.15.5:0/3351025249 conn(0x7f4c240b0d80 msgr2=0x7f4c240afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.682+0000 7f4c29a1c640 1 -- 172.21.15.5:0/3351025249 shutdown_connections 2023-11-24T06:16:01.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.682+0000 7f4c29a1c640 1 -- 172.21.15.5:0/3351025249 wait complete. 2023-11-24T06:16:01.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.6 2023-11-24T06:16:01.687 DEBUG:teuthology.parallel:result is None 2023-11-24T06:16:01.698 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.2 2023-11-24T06:16:01.699 DEBUG:teuthology.parallel:result is None 2023-11-24T06:16:01.710 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.3 2023-11-24T06:16:01.710 DEBUG:teuthology.parallel:result is None 2023-11-24T06:16:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.711+0000 7f2e8821c640 1 -- 172.21.15.5:0/112122632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2e80074770 con 0x7f2e800ae330 2023-11-24T06:16:01.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.711+0000 7f2e767fc640 1 -- 172.21.15.5:0/112122632 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2e7c0588c0 con 0x7f2e800ae330 2023-11-24T06:16:01.714 INFO:teuthology.orchestra.run.smithi005.stdout:64424509446 2023-11-24T06:16:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.713+0000 7f2e8821c640 1 -- 172.21.15.5:0/112122632 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2e6406ae90 msgr2=0x7f2e6406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.713+0000 7f2e8821c640 1 --2- 172.21.15.5:0/112122632 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2e6406ae90 0x7f2e6406d350 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2e80155c60 tx=0x7f2e78009040 comp rx=0 tx=0).stop 2023-11-24T06:16:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.713+0000 7f2e8821c640 1 -- 172.21.15.5:0/112122632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e800ae330 msgr2=0x7f2e800b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.713+0000 7f2e8821c640 1 --2- 172.21.15.5:0/112122632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e800ae330 0x7f2e800b0520 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c00d930 tx=0x7f2e7c00ddf0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.714+0000 7f2e8821c640 1 -- 172.21.15.5:0/112122632 shutdown_connections 2023-11-24T06:16:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.714+0000 7f2e8821c640 1 --2- 172.21.15.5:0/112122632 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2e6406ae90 0x7f2e6406d350 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.714+0000 7f2e8821c640 1 --2- 172.21.15.5:0/112122632 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2e80159d00 0x7f2e800b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.714+0000 7f2e8821c640 1 --2- 172.21.15.5:0/112122632 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2e8015abe0 0x7f2e800b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.714+0000 7f2e8821c640 1 --2- 172.21.15.5:0/112122632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e800ae330 0x7f2e800b0520 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.714+0000 7f2e8821c640 1 -- 172.21.15.5:0/112122632 >> 172.21.15.5:0/112122632 conn(0x7f2e800709f0 msgr2=0x7f2e8015a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.714+0000 7f2e8821c640 1 -- 172.21.15.5:0/112122632 shutdown_connections 2023-11-24T06:16:01.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.714+0000 7f2e8821c640 1 -- 172.21.15.5:0/112122632 wait complete. 2023-11-24T06:16:01.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.714+0000 7ffa0da31640 1 -- 172.21.15.5:0/2750653367 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ffa08074770 con 0x7ffa0806eb00 2023-11-24T06:16:01.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.715+0000 7ffa04ff9640 1 -- 172.21.15.5:0/2750653367 <== mon.1 v2:172.21.15.6:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff9f8058310 con 0x7ffa0806eb00 2023-11-24T06:16:01.718 INFO:teuthology.orchestra.run.smithi005.stdout:60129542152 2023-11-24T06:16:01.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.717+0000 7ffa0da31640 1 -- 172.21.15.5:0/2750653367 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff9e806ae40 msgr2=0x7ff9e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.717+0000 7ffa0da31640 1 --2- 172.21.15.5:0/2750653367 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff9e806ae40 0x7ff9e806d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffa081592d0 tx=0x7ff9f0008040 comp rx=0 tx=0).stop 2023-11-24T06:16:01.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.717+0000 7ffa0da31640 1 -- 172.21.15.5:0/2750653367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa0806eb00 msgr2=0x7ffa081f0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.717+0000 7ffa0da31640 1 --2- 172.21.15.5:0/2750653367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa0806eb00 0x7ffa081f0f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800efc0 tx=0x7ff9f800c490 comp rx=0 tx=0).stop 2023-11-24T06:16:01.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.717+0000 7ffa0da31640 1 -- 172.21.15.5:0/2750653367 shutdown_connections 2023-11-24T06:16:01.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.717+0000 7ffa0da31640 1 --2- 172.21.15.5:0/2750653367 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff9e806ae40 0x7ff9e806d300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.717+0000 7ffa0da31640 1 --2- 172.21.15.5:0/2750653367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffa080cfa00 0x7ffa08152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.717+0000 7ffa0da31640 1 --2- 172.21.15.5:0/2750653367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa080ce230 0x7ffa081551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.717+0000 7ffa0da31640 1 --2- 172.21.15.5:0/2750653367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffa0806eb00 0x7ffa081f0f30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.717+0000 7ffa0da31640 1 -- 172.21.15.5:0/2750653367 >> 172.21.15.5:0/2750653367 conn(0x7ffa080663d0 msgr2=0x7ffa080d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.718+0000 7ffa0da31640 1 -- 172.21.15.5:0/2750653367 shutdown_connections 2023-11-24T06:16:01.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.718+0000 7ffa0da31640 1 -- 172.21.15.5:0/2750653367 wait complete. 2023-11-24T06:16:01.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.719+0000 7f96857fa640 1 -- 172.21.15.5:0/3651229897 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9678005500 con 0x7f96a00d0230 2023-11-24T06:16:01.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.720+0000 7f96877fe640 1 -- 172.21.15.5:0/3651229897 <== mon.2 v2:172.21.15.6:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9698009510 con 0x7f96a00d0230 2023-11-24T06:16:01.723 INFO:teuthology.orchestra.run.smithi005.stdout:64424509445 2023-11-24T06:16:01.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.722+0000 7f96857fa640 1 -- 172.21.15.5:0/3651229897 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f968c06ae90 msgr2=0x7f968c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.722+0000 7f96857fa640 1 --2- 172.21.15.5:0/3651229897 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f968c06ae90 0x7f968c06d350 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f96a0148ff0 tx=0x7f9690009110 comp rx=0 tx=0).stop 2023-11-24T06:16:01.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.722+0000 7f96857fa640 1 -- 172.21.15.5:0/3651229897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96a00d0230 msgr2=0x7f96a0155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:01.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.722+0000 7f96857fa640 1 --2- 172.21.15.5:0/3651229897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96a00d0230 0x7f96a0155460 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9698000f80 tx=0x7f9698004a40 comp rx=0 tx=0).stop 2023-11-24T06:16:01.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.723+0000 7f96857fa640 1 -- 172.21.15.5:0/3651229897 shutdown_connections 2023-11-24T06:16:01.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.723+0000 7f96857fa640 1 --2- 172.21.15.5:0/3651229897 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f968c06ae90 0x7f968c06d350 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.723+0000 7f96857fa640 1 --2- 172.21.15.5:0/3651229897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a015d460 0x7f96a0086040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.723+0000 7f96857fa640 1 --2- 172.21.15.5:0/3651229897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96a00d1300 0x7f96a0149300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.723+0000 7f96857fa640 1 --2- 172.21.15.5:0/3651229897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96a00d0230 0x7f96a0155460 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:01.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.723+0000 7f96857fa640 1 -- 172.21.15.5:0/3651229897 >> 172.21.15.5:0/3651229897 conn(0x7f96a0071f90 msgr2=0x7f96a00b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:01.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.723+0000 7f96857fa640 1 -- 172.21.15.5:0/3651229897 shutdown_connections 2023-11-24T06:16:01.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.723+0000 7f96857fa640 1 -- 172.21.15.5:0/3651229897 wait complete. 2023-11-24T06:16:01.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.4 2023-11-24T06:16:01.735 DEBUG:teuthology.parallel:result is None 2023-11-24T06:16:01.737 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542152 for osd.5 2023-11-24T06:16:01.738 DEBUG:teuthology.parallel:result is None 2023-11-24T06:16:01.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.7 2023-11-24T06:16:01.742 DEBUG:teuthology.parallel:result is None 2023-11-24T06:16:01.742 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T06:16:01.742 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:16:02.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.998+0000 7fb6dffff640 1 Processor -- start 2023-11-24T06:16:02.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:01.998+0000 7fb6dffff640 1 -- start start 2023-11-24T06:16:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.000+0000 7fb6dffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6e0143570 0x7fb6e00cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.000+0000 7fb6dffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb6e0143950 0x7fb6e00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.000+0000 7fb6dffff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb6e015e6b0 0x7fb6e015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.000+0000 7fb6dffff640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e0071700 con 0x7fb6e0143950 2023-11-24T06:16:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.000+0000 7fb6dffff640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e015b410 con 0x7fb6e015e6b0 2023-11-24T06:16:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.000+0000 7fb6dffff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e015b130 con 0x7fb6e0143570 2023-11-24T06:16:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.000+0000 7fb6de7fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb6e0143950 0x7fb6e00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.000+0000 7fb6de7fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb6e0143950 0x7fb6e00cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36024/0 (socket says 172.21.15.5:36024) 2023-11-24T06:16:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.000+0000 7fb6de7fc640 1 -- 172.21.15.5:0/2328893712 learned_addr learned my addr 172.21.15.5:0/2328893712 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.000+0000 7fb6deffd640 1 --2- 172.21.15.5:0/2328893712 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6e0143570 0x7fb6e00cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.000+0000 7fb6df7fe640 1 --2- 172.21.15.5:0/2328893712 >> v2:172.21.15.6:3300/0 conn(0x7fb6e015e6b0 0x7fb6e015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.001+0000 7fb6de7fc640 1 -- 172.21.15.5:0/2328893712 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6e0143570 msgr2=0x7fb6e00cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.001+0000 7fb6de7fc640 1 --2- 172.21.15.5:0/2328893712 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6e0143570 0x7fb6e00cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.001+0000 7fb6de7fc640 1 -- 172.21.15.5:0/2328893712 >> v2:172.21.15.6:3300/0 conn(0x7fb6e015e6b0 msgr2=0x7fb6e015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.001+0000 7fb6de7fc640 1 --2- 172.21.15.5:0/2328893712 >> v2:172.21.15.6:3300/0 conn(0x7fb6e015e6b0 0x7fb6e015a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.001+0000 7fb6de7fc640 1 -- 172.21.15.5:0/2328893712 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e015dbd0 con 0x7fb6e0143950 2023-11-24T06:16:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.001+0000 7fb6deffd640 1 --2- 172.21.15.5:0/2328893712 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6e0143570 0x7fb6e00cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.002+0000 7fb6de7fc640 1 --2- 172.21.15.5:0/2328893712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e0143950 0x7fb6e00cf590 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0070ee0 tx=0x7fb6d402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe421dc8d4973c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.002+0000 7fb6ddffb640 1 -- 172.21.15.5:0/2328893712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40317e0 con 0x7fb6e0143950 2023-11-24T06:16:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.002+0000 7fb6ddffb640 1 -- 172.21.15.5:0/2328893712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d4031940 con 0x7fb6e0143950 2023-11-24T06:16:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.002+0000 7fb6ddffb640 1 -- 172.21.15.5:0/2328893712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4031cd0 con 0x7fb6e0143950 2023-11-24T06:16:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.003+0000 7fb6dffff640 1 -- 172.21.15.5:0/2328893712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e0143950 msgr2=0x7fb6e00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.003+0000 7fb6dffff640 1 --2- 172.21.15.5:0/2328893712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e0143950 0x7fb6e00cf590 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0070ee0 tx=0x7fb6d402b880 comp rx=0 tx=0).stop 2023-11-24T06:16:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.003+0000 7fb6dffff640 1 -- 172.21.15.5:0/2328893712 shutdown_connections 2023-11-24T06:16:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.003+0000 7fb6dffff640 1 --2- 172.21.15.5:0/2328893712 >> v2:172.21.15.6:3300/0 conn(0x7fb6e015e6b0 0x7fb6e015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.003+0000 7fb6dffff640 1 --2- 172.21.15.5:0/2328893712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e0143950 0x7fb6e00cf590 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.003+0000 7fb6dffff640 1 --2- 172.21.15.5:0/2328893712 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6e0143570 0x7fb6e00cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.003+0000 7fb6dffff640 1 -- 172.21.15.5:0/2328893712 >> 172.21.15.5:0/2328893712 conn(0x7fb6e006a860 msgr2=0x7fb6e006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:02.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.003+0000 7fb6dffff640 1 -- 172.21.15.5:0/2328893712 shutdown_connections 2023-11-24T06:16:02.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.003+0000 7fb6dffff640 1 -- 172.21.15.5:0/2328893712 wait complete. 2023-11-24T06:16:02.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.004+0000 7fb6dffff640 1 Processor -- start 2023-11-24T06:16:02.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.005+0000 7fb6dffff640 1 -- start start 2023-11-24T06:16:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.005+0000 7fb6dffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6e0143570 0x7fb6e0159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.005+0000 7fb6dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e0143950 0x7fb6e013fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.005+0000 7fb6dffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6e015e6b0 0x7fb6e01400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.005+0000 7fb6dffff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e013d380 con 0x7fb6e0143950 2023-11-24T06:16:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.005+0000 7fb6dffff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e013b9f0 con 0x7fb6e015e6b0 2023-11-24T06:16:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.005+0000 7fb6dffff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e013cae0 con 0x7fb6e0143570 2023-11-24T06:16:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.006+0000 7fb6de7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e0143950 0x7fb6e013fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.006+0000 7fb6de7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e0143950 0x7fb6e013fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36036/0 (socket says 172.21.15.5:36036) 2023-11-24T06:16:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.006+0000 7fb6de7fc640 1 -- 172.21.15.5:0/1947067927 learned_addr learned my addr 172.21.15.5:0/1947067927 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.006+0000 7fb6deffd640 1 --2- 172.21.15.5:0/1947067927 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6e0143570 0x7fb6e0159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.006+0000 7fb6df7fe640 1 --2- 172.21.15.5:0/1947067927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6e015e6b0 0x7fb6e01400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.006+0000 7fb6de7fc640 1 -- 172.21.15.5:0/1947067927 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6e0143570 msgr2=0x7fb6e0159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.006+0000 7fb6de7fc640 1 --2- 172.21.15.5:0/1947067927 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6e0143570 0x7fb6e0159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.006+0000 7fb6de7fc640 1 -- 172.21.15.5:0/1947067927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6e015e6b0 msgr2=0x7fb6e01400e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.006+0000 7fb6de7fc640 1 --2- 172.21.15.5:0/1947067927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6e015e6b0 0x7fb6e01400e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.006+0000 7fb6de7fc640 1 -- 172.21.15.5:0/1947067927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d40094d0 con 0x7fb6e0143950 2023-11-24T06:16:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.006+0000 7fb6de7fc640 1 --2- 172.21.15.5:0/1947067927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e0143950 0x7fb6e013fba0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4000c00 tx=0x7fb6d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.007+0000 7fb6c77fe640 1 -- 172.21.15.5:0/1947067927 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4032900 con 0x7fb6e0143950 2023-11-24T06:16:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.007+0000 7fb6c77fe640 1 -- 172.21.15.5:0/1947067927 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d4032a60 con 0x7fb6e0143950 2023-11-24T06:16:02.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.007+0000 7fb6c77fe640 1 -- 172.21.15.5:0/1947067927 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d4032e10 con 0x7fb6e0143950 2023-11-24T06:16:02.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.007+0000 7fb6dffff640 1 -- 172.21.15.5:0/1947067927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e013cdc0 con 0x7fb6e0143950 2023-11-24T06:16:02.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.007+0000 7fb6dffff640 1 -- 172.21.15.5:0/1947067927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e00c3be0 con 0x7fb6e0143950 2023-11-24T06:16:02.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.008+0000 7fb6dffff640 1 -- 172.21.15.5:0/1947067927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e00c3dd0 con 0x7fb6e0143950 2023-11-24T06:16:02.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.009+0000 7fb6c77fe640 1 -- 172.21.15.5:0/1947067927 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb6d404e450 con 0x7fb6e0143950 2023-11-24T06:16:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.010+0000 7fb6c77fe640 1 --2- 172.21.15.5:0/1947067927 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb6cc06ae40 0x7fb6cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.010+0000 7fb6c77fe640 1 -- 172.21.15.5:0/1947067927 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6d40ab920 con 0x7fb6e0143950 2023-11-24T06:16:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.010+0000 7fb6deffd640 1 --2- 172.21.15.5:0/1947067927 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb6cc06ae40 0x7fb6cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.011+0000 7fb6deffd640 1 --2- 172.21.15.5:0/1947067927 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb6cc06ae40 0x7fb6cc06d300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0008850 tx=0x7fb6d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:02.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.015+0000 7fb6c77fe640 1 -- 172.21.15.5:0/1947067927 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6d4076a70 con 0x7fb6e0143950 2023-11-24T06:16:02.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.196+0000 7fb6dffff640 1 -- 172.21.15.5:0/1947067927 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb6e015e190 con 0x7fb6cc06ae40 2023-11-24T06:16:02.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.205+0000 7fb6c77fe640 1 -- 172.21.15.5:0/1947067927 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264488 (secure 0 0 0) 0x7fb6d000c040 con 0x7fb6cc06ae40 2023-11-24T06:16:02.212 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:02.213 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-11-24T06:16:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.211+0000 7fb6dffff640 1 -- 172.21.15.5:0/1947067927 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb6cc06ae40 msgr2=0x7fb6cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.211+0000 7fb6dffff640 1 --2- 172.21.15.5:0/1947067927 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb6cc06ae40 0x7fb6cc06d300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0008850 tx=0x7fb6d0008040 comp rx=0 tx=0).stop 2023-11-24T06:16:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.211+0000 7fb6dffff640 1 -- 172.21.15.5:0/1947067927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e0143950 msgr2=0x7fb6e013fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.211+0000 7fb6dffff640 1 --2- 172.21.15.5:0/1947067927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e0143950 0x7fb6e013fba0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4000c00 tx=0x7fb6d40046c0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.211+0000 7fb6dffff640 1 -- 172.21.15.5:0/1947067927 shutdown_connections 2023-11-24T06:16:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.211+0000 7fb6dffff640 1 --2- 172.21.15.5:0/1947067927 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb6cc06ae40 0x7fb6cc06d300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.211+0000 7fb6dffff640 1 --2- 172.21.15.5:0/1947067927 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6e015e6b0 0x7fb6e01400e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.211+0000 7fb6dffff640 1 --2- 172.21.15.5:0/1947067927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6e0143950 0x7fb6e013fba0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.211+0000 7fb6dffff640 1 --2- 172.21.15.5:0/1947067927 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6e0143570 0x7fb6e0159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.211+0000 7fb6dffff640 1 -- 172.21.15.5:0/1947067927 >> 172.21.15.5:0/1947067927 conn(0x7fb6e006a860 msgr2=0x7fb6e00b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.212+0000 7fb6dffff640 1 -- 172.21.15.5:0/1947067927 shutdown_connections 2023-11-24T06:16:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.212+0000 7fb6dffff640 1 -- 172.21.15.5:0/1947067927 wait complete. 2023-11-24T06:16:02.233 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-24T06:16:02.090772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235764,"kb_used_data":2948,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738956,"statfs":{"total":773094113280,"available":772852690944,"internally_reserved":0,"allocated":3018752,"data_stored":1809589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003830"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185145+0000","last_change":"2023-11-24T06:15:51.826913+0000","last_active":"2023-11-24T06:15:54.185145+0000","last_peered":"2023-11-24T06:15:54.185145+0000","last_clean":"2023-11-24T06:15:54.185145+0000","last_became_active":"2023-11-24T06:15:51.826683+0000","last_became_peered":"2023-11-24T06:15:51.826683+0000","last_unstale":"2023-11-24T06:15:54.185145+0000","last_undegraded":"2023-11-24T06:15:54.185145+0000","last_fullsized":"2023-11-24T06:15:54.185145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:45:59.277130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789469+0000","last_change":"2023-11-24T06:15:51.820105+0000","last_active":"2023-11-24T06:15:52.789469+0000","last_peered":"2023-11-24T06:15:52.789469+0000","last_clean":"2023-11-24T06:15:52.789469+0000","last_became_active":"2023-11-24T06:15:51.819862+0000","last_became_peered":"2023-11-24T06:15:51.819862+0000","last_unstale":"2023-11-24T06:15:52.789469+0000","last_undegraded":"2023-11-24T06:15:52.789469+0000","last_fullsized":"2023-11-24T06:15:52.789469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:58.501546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.182835+0000","last_change":"2023-11-24T06:15:51.817345+0000","last_active":"2023-11-24T06:15:54.182835+0000","last_peered":"2023-11-24T06:15:54.182835+0000","last_clean":"2023-11-24T06:15:54.182835+0000","last_became_active":"2023-11-24T06:15:51.815455+0000","last_became_peered":"2023-11-24T06:15:51.815455+0000","last_unstale":"2023-11-24T06:15:54.182835+0000","last_undegraded":"2023-11-24T06:15:54.182835+0000","last_fullsized":"2023-11-24T06:15:54.182835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:50.460127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793545+0000","last_change":"2023-11-24T06:15:51.811576+0000","last_active":"2023-11-24T06:15:52.793545+0000","last_peered":"2023-11-24T06:15:52.793545+0000","last_clean":"2023-11-24T06:15:52.793545+0000","last_became_active":"2023-11-24T06:15:51.811306+0000","last_became_peered":"2023-11-24T06:15:51.811306+0000","last_unstale":"2023-11-24T06:15:52.793545+0000","last_undegraded":"2023-11-24T06:15:52.793545+0000","last_fullsized":"2023-11-24T06:15:52.793545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:34.521019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184624+0000","last_change":"2023-11-24T06:15:51.821890+0000","last_active":"2023-11-24T06:15:54.184624+0000","last_peered":"2023-11-24T06:15:54.184624+0000","last_clean":"2023-11-24T06:15:54.184624+0000","last_became_active":"2023-11-24T06:15:51.821632+0000","last_became_peered":"2023-11-24T06:15:51.821632+0000","last_unstale":"2023-11-24T06:15:54.184624+0000","last_undegraded":"2023-11-24T06:15:54.184624+0000","last_fullsized":"2023-11-24T06:15:54.184624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:54.384409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792886+0000","last_change":"2023-11-24T06:15:51.828326+0000","last_active":"2023-11-24T06:15:52.792886+0000","last_peered":"2023-11-24T06:15:52.792886+0000","last_clean":"2023-11-24T06:15:52.792886+0000","last_became_active":"2023-11-24T06:15:51.827922+0000","last_became_peered":"2023-11-24T06:15:51.827922+0000","last_unstale":"2023-11-24T06:15:52.792886+0000","last_undegraded":"2023-11-24T06:15:52.792886+0000","last_fullsized":"2023-11-24T06:15:52.792886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:25.825619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183858+0000","last_change":"2023-11-24T06:15:51.825625+0000","last_active":"2023-11-24T06:15:54.183858+0000","last_peered":"2023-11-24T06:15:54.183858+0000","last_clean":"2023-11-24T06:15:54.183858+0000","last_became_active":"2023-11-24T06:15:51.825308+0000","last_became_peered":"2023-11-24T06:15:51.825308+0000","last_unstale":"2023-11-24T06:15:54.183858+0000","last_undegraded":"2023-11-24T06:15:54.183858+0000","last_fullsized":"2023-11-24T06:15:54.183858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:58.501546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844647+0000","last_change":"2023-11-24T06:15:51.817175+0000","last_active":"2023-11-24T06:15:53.844647+0000","last_peered":"2023-11-24T06:15:53.844647+0000","last_clean":"2023-11-24T06:15:53.844647+0000","last_became_active":"2023-11-24T06:15:51.816363+0000","last_became_peered":"2023-11-24T06:15:51.816363+0000","last_unstale":"2023-11-24T06:15:53.844647+0000","last_undegraded":"2023-11-24T06:15:53.844647+0000","last_fullsized":"2023-11-24T06:15:53.844647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:00:51.554978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185142+0000","last_change":"2023-11-24T06:15:51.829604+0000","last_active":"2023-11-24T06:15:54.185142+0000","last_peered":"2023-11-24T06:15:54.185142+0000","last_clean":"2023-11-24T06:15:54.185142+0000","last_became_active":"2023-11-24T06:15:51.829291+0000","last_became_peered":"2023-11-24T06:15:51.829291+0000","last_unstale":"2023-11-24T06:15:54.185142+0000","last_undegraded":"2023-11-24T06:15:54.185142+0000","last_fullsized":"2023-11-24T06:15:54.185142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:57:53.020152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750691+0000","last_change":"2023-11-24T06:15:51.820996+0000","last_active":"2023-11-24T06:15:53.750691+0000","last_peered":"2023-11-24T06:15:53.750691+0000","last_clean":"2023-11-24T06:15:53.750691+0000","last_became_active":"2023-11-24T06:15:51.820018+0000","last_became_peered":"2023-11-24T06:15:51.820018+0000","last_unstale":"2023-11-24T06:15:53.750691+0000","last_undegraded":"2023-11-24T06:15:53.750691+0000","last_fullsized":"2023-11-24T06:15:53.750691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:05.782767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750915+0000","last_change":"2023-11-24T06:15:51.806762+0000","last_active":"2023-11-24T06:15:53.750915+0000","last_peered":"2023-11-24T06:15:53.750915+0000","last_clean":"2023-11-24T06:15:53.750915+0000","last_became_active":"2023-11-24T06:15:51.806476+0000","last_became_peered":"2023-11-24T06:15:51.806476+0000","last_unstale":"2023-11-24T06:15:53.750915+0000","last_undegraded":"2023-11-24T06:15:53.750915+0000","last_fullsized":"2023-11-24T06:15:53.750915+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:25.825619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791243+0000","last_change":"2023-11-24T06:15:51.807959+0000","last_active":"2023-11-24T06:15:52.791243+0000","last_peered":"2023-11-24T06:15:52.791243+0000","last_clean":"2023-11-24T06:15:52.791243+0000","last_became_active":"2023-11-24T06:15:51.807625+0000","last_became_peered":"2023-11-24T06:15:51.807625+0000","last_unstale":"2023-11-24T06:15:52.791243+0000","last_undegraded":"2023-11-24T06:15:52.791243+0000","last_fullsized":"2023-11-24T06:15:52.791243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:37.913398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845276+0000","last_change":"2023-11-24T06:15:51.820393+0000","last_active":"2023-11-24T06:15:53.845276+0000","last_peered":"2023-11-24T06:15:53.845276+0000","last_clean":"2023-11-24T06:15:53.845276+0000","last_became_active":"2023-11-24T06:15:51.820092+0000","last_became_peered":"2023-11-24T06:15:51.820092+0000","last_unstale":"2023-11-24T06:15:53.845276+0000","last_undegraded":"2023-11-24T06:15:53.845276+0000","last_fullsized":"2023-11-24T06:15:53.845276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:16.273773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.182517+0000","last_change":"2023-11-24T06:15:51.822418+0000","last_active":"2023-11-24T06:15:54.182517+0000","last_peered":"2023-11-24T06:15:54.182517+0000","last_clean":"2023-11-24T06:15:54.182517+0000","last_became_active":"2023-11-24T06:15:51.821162+0000","last_became_peered":"2023-11-24T06:15:51.821162+0000","last_unstale":"2023-11-24T06:15:54.182517+0000","last_undegraded":"2023-11-24T06:15:54.182517+0000","last_fullsized":"2023-11-24T06:15:54.182517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:23:44.726769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788580+0000","last_change":"2023-11-24T06:15:51.808930+0000","last_active":"2023-11-24T06:15:52.788580+0000","last_peered":"2023-11-24T06:15:52.788580+0000","last_clean":"2023-11-24T06:15:52.788580+0000","last_became_active":"2023-11-24T06:15:51.808593+0000","last_became_peered":"2023-11-24T06:15:51.808593+0000","last_unstale":"2023-11-24T06:15:52.788580+0000","last_undegraded":"2023-11-24T06:15:52.788580+0000","last_fullsized":"2023-11-24T06:15:52.788580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:43.291318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792334+0000","last_change":"2023-11-24T06:15:51.829868+0000","last_active":"2023-11-24T06:15:52.792334+0000","last_peered":"2023-11-24T06:15:52.792334+0000","last_clean":"2023-11-24T06:15:52.792334+0000","last_became_active":"2023-11-24T06:15:51.828875+0000","last_became_peered":"2023-11-24T06:15:51.828875+0000","last_unstale":"2023-11-24T06:15:52.792334+0000","last_undegraded":"2023-11-24T06:15:52.792334+0000","last_fullsized":"2023-11-24T06:15:52.792334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:32.225953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.794592+0000","last_change":"2023-11-24T06:15:51.814343+0000","last_active":"2023-11-24T06:15:52.794592+0000","last_peered":"2023-11-24T06:15:52.794592+0000","last_clean":"2023-11-24T06:15:52.794592+0000","last_became_active":"2023-11-24T06:15:51.814003+0000","last_became_peered":"2023-11-24T06:15:51.814003+0000","last_unstale":"2023-11-24T06:15:52.794592+0000","last_undegraded":"2023-11-24T06:15:52.794592+0000","last_fullsized":"2023-11-24T06:15:52.794592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:26:09.975671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.139925+0000","last_change":"2023-11-24T06:15:51.815884+0000","last_active":"2023-11-24T06:15:53.139925+0000","last_peered":"2023-11-24T06:15:53.139925+0000","last_clean":"2023-11-24T06:15:53.139925+0000","last_became_active":"2023-11-24T06:15:51.814549+0000","last_became_peered":"2023-11-24T06:15:51.814549+0000","last_unstale":"2023-11-24T06:15:53.139925+0000","last_undegraded":"2023-11-24T06:15:53.139925+0000","last_fullsized":"2023-11-24T06:15:53.139925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:43.354531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185458+0000","last_change":"2023-11-24T06:15:51.823265+0000","last_active":"2023-11-24T06:15:54.185458+0000","last_peered":"2023-11-24T06:15:54.185458+0000","last_clean":"2023-11-24T06:15:54.185458+0000","last_became_active":"2023-11-24T06:15:51.823026+0000","last_became_peered":"2023-11-24T06:15:51.823026+0000","last_unstale":"2023-11-24T06:15:54.185458+0000","last_undegraded":"2023-11-24T06:15:54.185458+0000","last_fullsized":"2023-11-24T06:15:54.185458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:48.210729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750151+0000","last_change":"2023-11-24T06:15:51.818614+0000","last_active":"2023-11-24T06:15:53.750151+0000","last_peered":"2023-11-24T06:15:53.750151+0000","last_clean":"2023-11-24T06:15:53.750151+0000","last_became_active":"2023-11-24T06:15:51.818250+0000","last_became_peered":"2023-11-24T06:15:51.818250+0000","last_unstale":"2023-11-24T06:15:53.750151+0000","last_undegraded":"2023-11-24T06:15:53.750151+0000","last_fullsized":"2023-11-24T06:15:53.750151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:07.460859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.182802+0000","last_change":"2023-11-24T06:15:51.815798+0000","last_active":"2023-11-24T06:15:54.182802+0000","last_peered":"2023-11-24T06:15:54.182802+0000","last_clean":"2023-11-24T06:15:54.182802+0000","last_became_active":"2023-11-24T06:15:51.815531+0000","last_became_peered":"2023-11-24T06:15:51.815531+0000","last_unstale":"2023-11-24T06:15:54.182802+0000","last_undegraded":"2023-11-24T06:15:54.182802+0000","last_fullsized":"2023-11-24T06:15:54.182802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:55.713409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.794291+0000","last_change":"2023-11-24T06:15:51.813391+0000","last_active":"2023-11-24T06:15:52.794291+0000","last_peered":"2023-11-24T06:15:52.794291+0000","last_clean":"2023-11-24T06:15:52.794291+0000","last_became_active":"2023-11-24T06:15:51.813066+0000","last_became_peered":"2023-11-24T06:15:51.813066+0000","last_unstale":"2023-11-24T06:15:52.794291+0000","last_undegraded":"2023-11-24T06:15:52.794291+0000","last_fullsized":"2023-11-24T06:15:52.794291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:54:56.970614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844721+0000","last_change":"2023-11-24T06:15:51.812820+0000","last_active":"2023-11-24T06:15:53.844721+0000","last_peered":"2023-11-24T06:15:53.844721+0000","last_clean":"2023-11-24T06:15:53.844721+0000","last_became_active":"2023-11-24T06:15:51.812517+0000","last_became_peered":"2023-11-24T06:15:51.812517+0000","last_unstale":"2023-11-24T06:15:53.844721+0000","last_undegraded":"2023-11-24T06:15:53.844721+0000","last_fullsized":"2023-11-24T06:15:53.844721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:25.102989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792262+0000","last_change":"2023-11-24T06:15:51.808797+0000","last_active":"2023-11-24T06:15:52.792262+0000","last_peered":"2023-11-24T06:15:52.792262+0000","last_clean":"2023-11-24T06:15:52.792262+0000","last_became_active":"2023-11-24T06:15:51.808481+0000","last_became_peered":"2023-11-24T06:15:51.808481+0000","last_unstale":"2023-11-24T06:15:52.792262+0000","last_undegraded":"2023-11-24T06:15:52.792262+0000","last_fullsized":"2023-11-24T06:15:52.792262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:05.782767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845642+0000","last_change":"2023-11-24T06:15:51.818331+0000","last_active":"2023-11-24T06:15:53.845642+0000","last_peered":"2023-11-24T06:15:53.845642+0000","last_clean":"2023-11-24T06:15:53.845642+0000","last_became_active":"2023-11-24T06:15:51.818011+0000","last_became_peered":"2023-11-24T06:15:51.818011+0000","last_unstale":"2023-11-24T06:15:53.845642+0000","last_undegraded":"2023-11-24T06:15:53.845642+0000","last_fullsized":"2023-11-24T06:15:53.845642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:58.501546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750817+0000","last_change":"2023-11-24T06:15:51.819933+0000","last_active":"2023-11-24T06:15:53.750817+0000","last_peered":"2023-11-24T06:15:53.750817+0000","last_clean":"2023-11-24T06:15:53.750817+0000","last_became_active":"2023-11-24T06:15:51.819592+0000","last_became_peered":"2023-11-24T06:15:51.819592+0000","last_unstale":"2023-11-24T06:15:53.750817+0000","last_undegraded":"2023-11-24T06:15:53.750817+0000","last_fullsized":"2023-11-24T06:15:53.750817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:32.225953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788974+0000","last_change":"2023-11-24T06:15:51.810396+0000","last_active":"2023-11-24T06:15:52.788974+0000","last_peered":"2023-11-24T06:15:52.788974+0000","last_clean":"2023-11-24T06:15:52.788974+0000","last_became_active":"2023-11-24T06:15:51.809939+0000","last_became_peered":"2023-11-24T06:15:51.809939+0000","last_unstale":"2023-11-24T06:15:52.788974+0000","last_undegraded":"2023-11-24T06:15:52.788974+0000","last_fullsized":"2023-11-24T06:15:52.788974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:16.273773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845858+0000","last_change":"2023-11-24T06:15:51.818877+0000","last_active":"2023-11-24T06:15:53.845858+0000","last_peered":"2023-11-24T06:15:53.845858+0000","last_clean":"2023-11-24T06:15:53.845858+0000","last_became_active":"2023-11-24T06:15:51.818068+0000","last_became_peered":"2023-11-24T06:15:51.818068+0000","last_unstale":"2023-11-24T06:15:53.845858+0000","last_undegraded":"2023-11-24T06:15:53.845858+0000","last_fullsized":"2023-11-24T06:15:53.845858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:54.384409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.182872+0000","last_change":"2023-11-24T06:15:51.821479+0000","last_active":"2023-11-24T06:15:54.182872+0000","last_peered":"2023-11-24T06:15:54.182872+0000","last_clean":"2023-11-24T06:15:54.182872+0000","last_became_active":"2023-11-24T06:15:51.820978+0000","last_became_peered":"2023-11-24T06:15:51.820978+0000","last_unstale":"2023-11-24T06:15:54.182872+0000","last_undegraded":"2023-11-24T06:15:54.182872+0000","last_fullsized":"2023-11-24T06:15:54.182872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:57:57.197250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793272+0000","last_change":"2023-11-24T06:15:51.808157+0000","last_active":"2023-11-24T06:15:52.793272+0000","last_peered":"2023-11-24T06:15:52.793272+0000","last_clean":"2023-11-24T06:15:52.793272+0000","last_became_active":"2023-11-24T06:15:51.807868+0000","last_became_peered":"2023-11-24T06:15:51.807868+0000","last_unstale":"2023-11-24T06:15:52.793272+0000","last_undegraded":"2023-11-24T06:15:52.793272+0000","last_fullsized":"2023-11-24T06:15:52.793272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:41:05.120915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.140134+0000","last_change":"2023-11-24T06:15:51.809534+0000","last_active":"2023-11-24T06:15:53.140134+0000","last_peered":"2023-11-24T06:15:53.140134+0000","last_clean":"2023-11-24T06:15:53.140134+0000","last_became_active":"2023-11-24T06:15:51.809037+0000","last_became_peered":"2023-11-24T06:15:51.809037+0000","last_unstale":"2023-11-24T06:15:53.140134+0000","last_undegraded":"2023-11-24T06:15:53.140134+0000","last_fullsized":"2023-11-24T06:15:53.140134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:38.032032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750282+0000","last_change":"2023-11-24T06:15:51.825619+0000","last_active":"2023-11-24T06:15:53.750282+0000","last_peered":"2023-11-24T06:15:53.750282+0000","last_clean":"2023-11-24T06:15:53.750282+0000","last_became_active":"2023-11-24T06:15:51.825149+0000","last_became_peered":"2023-11-24T06:15:51.825149+0000","last_unstale":"2023-11-24T06:15:53.750282+0000","last_undegraded":"2023-11-24T06:15:53.750282+0000","last_fullsized":"2023-11-24T06:15:53.750282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:12:12.520019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791884+0000","last_change":"2023-11-24T06:15:51.810514+0000","last_active":"2023-11-24T06:15:52.791884+0000","last_peered":"2023-11-24T06:15:52.791884+0000","last_clean":"2023-11-24T06:15:52.791884+0000","last_became_active":"2023-11-24T06:15:51.807943+0000","last_became_peered":"2023-11-24T06:15:51.807943+0000","last_unstale":"2023-11-24T06:15:52.791884+0000","last_undegraded":"2023-11-24T06:15:52.791884+0000","last_fullsized":"2023-11-24T06:15:52.791884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:19.993397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845626+0000","last_change":"2023-11-24T06:15:51.164865+0000","last_active":"2023-11-24T06:15:53.845626+0000","last_peered":"2023-11-24T06:15:53.845626+0000","last_clean":"2023-11-24T06:15:53.845626+0000","last_became_active":"2023-11-24T06:15:51.164524+0000","last_became_peered":"2023-11-24T06:15:51.164524+0000","last_unstale":"2023-11-24T06:15:53.845626+0000","last_undegraded":"2023-11-24T06:15:53.845626+0000","last_fullsized":"2023-11-24T06:15:53.845626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:02.689172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183027+0000","last_change":"2023-11-24T06:15:51.818737+0000","last_active":"2023-11-24T06:15:54.183027+0000","last_peered":"2023-11-24T06:15:54.183027+0000","last_clean":"2023-11-24T06:15:54.183027+0000","last_became_active":"2023-11-24T06:15:51.817354+0000","last_became_peered":"2023-11-24T06:15:51.817354+0000","last_unstale":"2023-11-24T06:15:54.183027+0000","last_undegraded":"2023-11-24T06:15:54.183027+0000","last_fullsized":"2023-11-24T06:15:54.183027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:07.460859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749981+0000","last_change":"2023-11-24T06:15:51.169166+0000","last_active":"2023-11-24T06:15:53.749981+0000","last_peered":"2023-11-24T06:15:53.749981+0000","last_clean":"2023-11-24T06:15:53.749981+0000","last_became_active":"2023-11-24T06:15:51.168863+0000","last_became_peered":"2023-11-24T06:15:51.168863+0000","last_unstale":"2023-11-24T06:15:53.749981+0000","last_undegraded":"2023-11-24T06:15:53.749981+0000","last_fullsized":"2023-11-24T06:15:53.749981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:16.146869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844773+0000","last_change":"2023-11-24T06:15:51.165957+0000","last_active":"2023-11-24T06:15:53.844773+0000","last_peered":"2023-11-24T06:15:53.844773+0000","last_clean":"2023-11-24T06:15:53.844773+0000","last_became_active":"2023-11-24T06:15:51.165575+0000","last_became_peered":"2023-11-24T06:15:51.165575+0000","last_unstale":"2023-11-24T06:15:53.844773+0000","last_undegraded":"2023-11-24T06:15:53.844773+0000","last_fullsized":"2023-11-24T06:15:53.844773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:09.461899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844747+0000","last_change":"2023-11-24T06:15:51.818508+0000","last_active":"2023-11-24T06:15:53.844747+0000","last_peered":"2023-11-24T06:15:53.844747+0000","last_clean":"2023-11-24T06:15:53.844747+0000","last_became_active":"2023-11-24T06:15:51.818158+0000","last_became_peered":"2023-11-24T06:15:51.818158+0000","last_unstale":"2023-11-24T06:15:53.844747+0000","last_undegraded":"2023-11-24T06:15:53.844747+0000","last_fullsized":"2023-11-24T06:15:53.844747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:43.291318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792534+0000","last_change":"2023-11-24T06:15:51.168437+0000","last_active":"2023-11-24T06:15:52.792534+0000","last_peered":"2023-11-24T06:15:52.792534+0000","last_clean":"2023-11-24T06:15:52.792534+0000","last_became_active":"2023-11-24T06:15:51.162079+0000","last_became_peered":"2023-11-24T06:15:51.162079+0000","last_unstale":"2023-11-24T06:15:52.792534+0000","last_undegraded":"2023-11-24T06:15:52.792534+0000","last_fullsized":"2023-11-24T06:15:52.792534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:07.834230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788318+0000","last_change":"2023-11-24T06:15:51.811023+0000","last_active":"2023-11-24T06:15:52.788318+0000","last_peered":"2023-11-24T06:15:52.788318+0000","last_clean":"2023-11-24T06:15:52.788318+0000","last_became_active":"2023-11-24T06:15:51.810368+0000","last_became_peered":"2023-11-24T06:15:51.810368+0000","last_unstale":"2023-11-24T06:15:52.788318+0000","last_undegraded":"2023-11-24T06:15:52.788318+0000","last_fullsized":"2023-11-24T06:15:52.788318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:25.102989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750531+0000","last_change":"2023-11-24T06:15:51.175246+0000","last_active":"2023-11-24T06:15:53.750531+0000","last_peered":"2023-11-24T06:15:53.750531+0000","last_clean":"2023-11-24T06:15:53.750531+0000","last_became_active":"2023-11-24T06:15:51.174941+0000","last_became_peered":"2023-11-24T06:15:51.174941+0000","last_unstale":"2023-11-24T06:15:53.750531+0000","last_undegraded":"2023-11-24T06:15:53.750531+0000","last_fullsized":"2023-11-24T06:15:53.750531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:48.317425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793290+0000","last_change":"2023-11-24T06:15:51.822074+0000","last_active":"2023-11-24T06:15:52.793290+0000","last_peered":"2023-11-24T06:15:52.793290+0000","last_clean":"2023-11-24T06:15:52.793290+0000","last_became_active":"2023-11-24T06:15:51.821749+0000","last_became_peered":"2023-11-24T06:15:51.821749+0000","last_unstale":"2023-11-24T06:15:52.793290+0000","last_undegraded":"2023-11-24T06:15:52.793290+0000","last_fullsized":"2023-11-24T06:15:52.793290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:47:30.989438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793436+0000","last_change":"2023-11-24T06:15:51.175128+0000","last_active":"2023-11-24T06:15:52.793436+0000","last_peered":"2023-11-24T06:15:52.793436+0000","last_clean":"2023-11-24T06:15:52.793436+0000","last_became_active":"2023-11-24T06:15:51.173422+0000","last_became_peered":"2023-11-24T06:15:51.173422+0000","last_unstale":"2023-11-24T06:15:52.793436+0000","last_undegraded":"2023-11-24T06:15:52.793436+0000","last_fullsized":"2023-11-24T06:15:52.793436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:12:11.507962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183145+0000","last_change":"2023-11-24T06:15:51.814210+0000","last_active":"2023-11-24T06:15:54.183145+0000","last_peered":"2023-11-24T06:15:54.183145+0000","last_clean":"2023-11-24T06:15:54.183145+0000","last_became_active":"2023-11-24T06:15:51.813951+0000","last_became_peered":"2023-11-24T06:15:51.813951+0000","last_unstale":"2023-11-24T06:15:54.183145+0000","last_undegraded":"2023-11-24T06:15:54.183145+0000","last_fullsized":"2023-11-24T06:15:54.183145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:32.225953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789082+0000","last_change":"2023-11-24T06:15:51.176266+0000","last_active":"2023-11-24T06:15:52.789082+0000","last_peered":"2023-11-24T06:15:52.789082+0000","last_clean":"2023-11-24T06:15:52.789082+0000","last_became_active":"2023-11-24T06:15:51.175918+0000","last_became_peered":"2023-11-24T06:15:51.175918+0000","last_unstale":"2023-11-24T06:15:52.789082+0000","last_undegraded":"2023-11-24T06:15:52.789082+0000","last_fullsized":"2023-11-24T06:15:52.789082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:18.981340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791650+0000","last_change":"2023-11-24T06:15:51.819556+0000","last_active":"2023-11-24T06:15:52.791650+0000","last_peered":"2023-11-24T06:15:52.791650+0000","last_clean":"2023-11-24T06:15:52.791650+0000","last_became_active":"2023-11-24T06:15:51.818398+0000","last_became_peered":"2023-11-24T06:15:51.818398+0000","last_unstale":"2023-11-24T06:15:52.791650+0000","last_undegraded":"2023-11-24T06:15:52.791650+0000","last_fullsized":"2023-11-24T06:15:52.791650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:41.065020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750080+0000","last_change":"2023-11-24T06:15:51.169595+0000","last_active":"2023-11-24T06:15:53.750080+0000","last_peered":"2023-11-24T06:15:53.750080+0000","last_clean":"2023-11-24T06:15:53.750080+0000","last_became_active":"2023-11-24T06:15:51.167755+0000","last_became_peered":"2023-11-24T06:15:51.167755+0000","last_unstale":"2023-11-24T06:15:53.750080+0000","last_undegraded":"2023-11-24T06:15:53.750080+0000","last_fullsized":"2023-11-24T06:15:53.750080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:37.019975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184774+0000","last_change":"2023-11-24T06:15:51.821305+0000","last_active":"2023-11-24T06:15:54.184774+0000","last_peered":"2023-11-24T06:15:54.184774+0000","last_clean":"2023-11-24T06:15:54.184774+0000","last_became_active":"2023-11-24T06:15:51.821061+0000","last_became_peered":"2023-11-24T06:15:51.821061+0000","last_unstale":"2023-11-24T06:15:54.184774+0000","last_undegraded":"2023-11-24T06:15:54.184774+0000","last_fullsized":"2023-11-24T06:15:54.184774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:24.298719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.129081+0000","last_change":"2023-11-24T06:15:51.150730+0000","last_active":"2023-11-24T06:15:54.129081+0000","last_peered":"2023-11-24T06:15:54.129081+0000","last_clean":"2023-11-24T06:15:54.129081+0000","last_became_active":"2023-11-24T06:15:51.147375+0000","last_became_peered":"2023-11-24T06:15:51.147375+0000","last_unstale":"2023-11-24T06:15:54.129081+0000","last_undegraded":"2023-11-24T06:15:54.129081+0000","last_fullsized":"2023-11-24T06:15:54.129081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:24:38.346475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.182531+0000","last_change":"2023-11-24T06:15:51.822499+0000","last_active":"2023-11-24T06:15:54.182531+0000","last_peered":"2023-11-24T06:15:54.182531+0000","last_clean":"2023-11-24T06:15:54.182531+0000","last_became_active":"2023-11-24T06:15:51.822192+0000","last_became_peered":"2023-11-24T06:15:51.822192+0000","last_unstale":"2023-11-24T06:15:54.182531+0000","last_undegraded":"2023-11-24T06:15:54.182531+0000","last_fullsized":"2023-11-24T06:15:54.182531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:49.329482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793561+0000","last_change":"2023-11-24T06:15:51.174234+0000","last_active":"2023-11-24T06:15:52.793561+0000","last_peered":"2023-11-24T06:15:52.793561+0000","last_clean":"2023-11-24T06:15:52.793561+0000","last_became_active":"2023-11-24T06:15:51.173741+0000","last_became_peered":"2023-11-24T06:15:51.173741+0000","last_unstale":"2023-11-24T06:15:52.793561+0000","last_undegraded":"2023-11-24T06:15:52.793561+0000","last_fullsized":"2023-11-24T06:15:52.793561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:16.146869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184654+0000","last_change":"2023-11-24T06:15:51.828660+0000","last_active":"2023-11-24T06:15:54.184654+0000","last_peered":"2023-11-24T06:15:54.184654+0000","last_clean":"2023-11-24T06:15:54.184654+0000","last_became_active":"2023-11-24T06:15:51.828176+0000","last_became_peered":"2023-11-24T06:15:51.828176+0000","last_unstale":"2023-11-24T06:15:54.184654+0000","last_undegraded":"2023-11-24T06:15:54.184654+0000","last_fullsized":"2023-11-24T06:15:54.184654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:47.291087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749753+0000","last_change":"2023-11-24T06:15:51.171268+0000","last_active":"2023-11-24T06:15:53.749753+0000","last_peered":"2023-11-24T06:15:53.749753+0000","last_clean":"2023-11-24T06:15:53.749753+0000","last_became_active":"2023-11-24T06:15:51.170887+0000","last_became_peered":"2023-11-24T06:15:51.170887+0000","last_unstale":"2023-11-24T06:15:53.749753+0000","last_undegraded":"2023-11-24T06:15:53.749753+0000","last_fullsized":"2023-11-24T06:15:53.749753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:24:38.346475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749967+0000","last_change":"2023-11-24T06:15:51.825705+0000","last_active":"2023-11-24T06:15:53.749967+0000","last_peered":"2023-11-24T06:15:53.749967+0000","last_clean":"2023-11-24T06:15:53.749967+0000","last_became_active":"2023-11-24T06:15:51.825372+0000","last_became_peered":"2023-11-24T06:15:51.825372+0000","last_unstale":"2023-11-24T06:15:53.749967+0000","last_undegraded":"2023-11-24T06:15:53.749967+0000","last_fullsized":"2023-11-24T06:15:53.749967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:57:57.197250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.139651+0000","last_change":"2023-11-24T06:15:51.155610+0000","last_active":"2023-11-24T06:15:53.139651+0000","last_peered":"2023-11-24T06:15:53.139651+0000","last_clean":"2023-11-24T06:15:53.139651+0000","last_became_active":"2023-11-24T06:15:51.155318+0000","last_became_peered":"2023-11-24T06:15:51.155318+0000","last_unstale":"2023-11-24T06:15:53.139651+0000","last_undegraded":"2023-11-24T06:15:53.139651+0000","last_fullsized":"2023-11-24T06:15:53.139651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:07.193249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183869+0000","last_change":"2023-11-24T06:15:51.825462+0000","last_active":"2023-11-24T06:15:54.183869+0000","last_peered":"2023-11-24T06:15:54.183869+0000","last_clean":"2023-11-24T06:15:54.183869+0000","last_became_active":"2023-11-24T06:15:51.825148+0000","last_became_peered":"2023-11-24T06:15:51.825148+0000","last_unstale":"2023-11-24T06:15:54.183869+0000","last_undegraded":"2023-11-24T06:15:54.183869+0000","last_fullsized":"2023-11-24T06:15:54.183869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:46.484813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.158814+0000","last_change":"2023-11-24T06:15:51.151498+0000","last_active":"2023-11-24T06:15:52.158814+0000","last_peered":"2023-11-24T06:15:52.158814+0000","last_clean":"2023-11-24T06:15:52.158814+0000","last_became_active":"2023-11-24T06:15:51.151116+0000","last_became_peered":"2023-11-24T06:15:51.151116+0000","last_unstale":"2023-11-24T06:15:52.158814+0000","last_undegraded":"2023-11-24T06:15:52.158814+0000","last_fullsized":"2023-11-24T06:15:52.158814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:32.153844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184490+0000","last_change":"2023-11-24T06:15:51.826667+0000","last_active":"2023-11-24T06:15:54.184490+0000","last_peered":"2023-11-24T06:15:54.184490+0000","last_clean":"2023-11-24T06:15:54.184490+0000","last_became_active":"2023-11-24T06:15:51.826286+0000","last_became_peered":"2023-11-24T06:15:51.826286+0000","last_unstale":"2023-11-24T06:15:54.184490+0000","last_undegraded":"2023-11-24T06:15:54.184490+0000","last_fullsized":"2023-11-24T06:15:54.184490+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:27.816377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.140108+0000","last_change":"2023-11-24T06:15:51.156776+0000","last_active":"2023-11-24T06:15:53.140108+0000","last_peered":"2023-11-24T06:15:53.140108+0000","last_clean":"2023-11-24T06:15:53.140108+0000","last_became_active":"2023-11-24T06:15:51.156313+0000","last_became_peered":"2023-11-24T06:15:51.156313+0000","last_unstale":"2023-11-24T06:15:53.140108+0000","last_undegraded":"2023-11-24T06:15:53.140108+0000","last_fullsized":"2023-11-24T06:15:53.140108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:47:58.118634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185599+0000","last_change":"2023-11-24T06:15:51.825443+0000","last_active":"2023-11-24T06:15:54.185599+0000","last_peered":"2023-11-24T06:15:54.185599+0000","last_clean":"2023-11-24T06:15:54.185599+0000","last_became_active":"2023-11-24T06:15:51.825149+0000","last_became_peered":"2023-11-24T06:15:51.825149+0000","last_unstale":"2023-11-24T06:15:54.185599+0000","last_undegraded":"2023-11-24T06:15:54.185599+0000","last_fullsized":"2023-11-24T06:15:54.185599+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:37.832748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789200+0000","last_change":"2023-11-24T06:15:51.158822+0000","last_active":"2023-11-24T06:15:52.789200+0000","last_peered":"2023-11-24T06:15:52.789200+0000","last_clean":"2023-11-24T06:15:52.789200+0000","last_became_active":"2023-11-24T06:15:51.158544+0000","last_became_peered":"2023-11-24T06:15:51.158544+0000","last_unstale":"2023-11-24T06:15:52.789200+0000","last_undegraded":"2023-11-24T06:15:52.789200+0000","last_fullsized":"2023-11-24T06:15:52.789200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:02.689172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.794289+0000","last_change":"2023-11-24T06:15:51.808175+0000","last_active":"2023-11-24T06:15:52.794289+0000","last_peered":"2023-11-24T06:15:52.794289+0000","last_clean":"2023-11-24T06:15:52.794289+0000","last_became_active":"2023-11-24T06:15:51.807904+0000","last_became_peered":"2023-11-24T06:15:51.807904+0000","last_unstale":"2023-11-24T06:15:52.794289+0000","last_undegraded":"2023-11-24T06:15:52.794289+0000","last_fullsized":"2023-11-24T06:15:52.794289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:54.525459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.139914+0000","last_change":"2023-11-24T06:15:51.175398+0000","last_active":"2023-11-24T06:15:53.139914+0000","last_peered":"2023-11-24T06:15:53.139914+0000","last_clean":"2023-11-24T06:15:53.139914+0000","last_became_active":"2023-11-24T06:15:51.175039+0000","last_became_peered":"2023-11-24T06:15:51.175039+0000","last_unstale":"2023-11-24T06:15:53.139914+0000","last_undegraded":"2023-11-24T06:15:53.139914+0000","last_fullsized":"2023-11-24T06:15:53.139914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:07.834230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185074+0000","last_change":"2023-11-24T06:15:51.829803+0000","last_active":"2023-11-24T06:15:54.185074+0000","last_peered":"2023-11-24T06:15:54.185074+0000","last_clean":"2023-11-24T06:15:54.185074+0000","last_became_active":"2023-11-24T06:15:51.829417+0000","last_became_peered":"2023-11-24T06:15:51.829417+0000","last_unstale":"2023-11-24T06:15:54.185074+0000","last_undegraded":"2023-11-24T06:15:54.185074+0000","last_fullsized":"2023-11-24T06:15:54.185074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:55:04.397371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.810683+0000","last_change":"2023-11-24T06:15:42.021608+0000","last_active":"2023-11-24T06:15:51.810683+0000","last_peered":"2023-11-24T06:15:51.810683+0000","last_clean":"2023-11-24T06:15:51.810683+0000","last_became_active":"2023-11-24T06:15:42.018847+0000","last_became_peered":"2023-11-24T06:15:42.018847+0000","last_unstale":"2023-11-24T06:15:51.810683+0000","last_undegraded":"2023-11-24T06:15:51.810683+0000","last_fullsized":"2023-11-24T06:15:51.810683+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_clean_scrub_stamp":"2023-11-24T06:15:40.694368+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:41.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792437+0000","last_change":"2023-11-24T06:15:51.161936+0000","last_active":"2023-11-24T06:15:52.792437+0000","last_peered":"2023-11-24T06:15:52.792437+0000","last_clean":"2023-11-24T06:15:52.792437+0000","last_became_active":"2023-11-24T06:15:51.161566+0000","last_became_peered":"2023-11-24T06:15:51.161566+0000","last_unstale":"2023-11-24T06:15:52.792437+0000","last_undegraded":"2023-11-24T06:15:52.792437+0000","last_fullsized":"2023-11-24T06:15:52.792437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:42.342474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845108+0000","last_change":"2023-11-24T06:15:51.816547+0000","last_active":"2023-11-24T06:15:53.845108+0000","last_peered":"2023-11-24T06:15:53.845108+0000","last_clean":"2023-11-24T06:15:53.845108+0000","last_became_active":"2023-11-24T06:15:51.816231+0000","last_became_peered":"2023-11-24T06:15:51.816231+0000","last_unstale":"2023-11-24T06:15:53.845108+0000","last_undegraded":"2023-11-24T06:15:53.845108+0000","last_fullsized":"2023-11-24T06:15:53.845108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:21:49.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183719+0000","last_change":"2023-11-24T06:15:51.164362+0000","last_active":"2023-11-24T06:15:54.183719+0000","last_peered":"2023-11-24T06:15:54.183719+0000","last_clean":"2023-11-24T06:15:54.183719+0000","last_became_active":"2023-11-24T06:15:51.163606+0000","last_became_peered":"2023-11-24T06:15:51.163606+0000","last_unstale":"2023-11-24T06:15:54.183719+0000","last_undegraded":"2023-11-24T06:15:54.183719+0000","last_fullsized":"2023-11-24T06:15:54.183719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:36.901341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.182391+0000","last_change":"2023-11-24T06:15:51.821396+0000","last_active":"2023-11-24T06:15:54.182391+0000","last_peered":"2023-11-24T06:15:54.182391+0000","last_clean":"2023-11-24T06:15:54.182391+0000","last_became_active":"2023-11-24T06:15:51.821096+0000","last_became_peered":"2023-11-24T06:15:51.821096+0000","last_unstale":"2023-11-24T06:15:54.182391+0000","last_undegraded":"2023-11-24T06:15:54.182391+0000","last_fullsized":"2023-11-24T06:15:54.182391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:17.158925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.140207+0000","last_change":"2023-11-24T06:15:51.156378+0000","last_active":"2023-11-24T06:15:53.140207+0000","last_peered":"2023-11-24T06:15:53.140207+0000","last_clean":"2023-11-24T06:15:53.140207+0000","last_became_active":"2023-11-24T06:15:51.156003+0000","last_became_peered":"2023-11-24T06:15:51.156003+0000","last_unstale":"2023-11-24T06:15:53.140207+0000","last_undegraded":"2023-11-24T06:15:53.140207+0000","last_fullsized":"2023-11-24T06:15:53.140207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:31.267818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788625+0000","last_change":"2023-11-24T06:15:51.810053+0000","last_active":"2023-11-24T06:15:52.788625+0000","last_peered":"2023-11-24T06:15:52.788625+0000","last_clean":"2023-11-24T06:15:52.788625+0000","last_became_active":"2023-11-24T06:15:51.806554+0000","last_became_peered":"2023-11-24T06:15:51.806554+0000","last_unstale":"2023-11-24T06:15:52.788625+0000","last_undegraded":"2023-11-24T06:15:52.788625+0000","last_fullsized":"2023-11-24T06:15:52.788625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:01.976941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793809+0000","last_change":"2023-11-24T06:15:51.170982+0000","last_active":"2023-11-24T06:15:52.793809+0000","last_peered":"2023-11-24T06:15:52.793809+0000","last_clean":"2023-11-24T06:15:52.793809+0000","last_became_active":"2023-11-24T06:15:51.170654+0000","last_became_peered":"2023-11-24T06:15:51.170654+0000","last_unstale":"2023-11-24T06:15:52.793809+0000","last_undegraded":"2023-11-24T06:15:52.793809+0000","last_fullsized":"2023-11-24T06:15:52.793809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:48.317425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792676+0000","last_change":"2023-11-24T06:15:51.823915+0000","last_active":"2023-11-24T06:15:52.792676+0000","last_peered":"2023-11-24T06:15:52.792676+0000","last_clean":"2023-11-24T06:15:52.792676+0000","last_became_active":"2023-11-24T06:15:51.823654+0000","last_became_peered":"2023-11-24T06:15:51.823654+0000","last_unstale":"2023-11-24T06:15:52.792676+0000","last_undegraded":"2023-11-24T06:15:52.792676+0000","last_fullsized":"2023-11-24T06:15:52.792676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:17:01.717621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789034+0000","last_change":"2023-11-24T06:15:51.154171+0000","last_active":"2023-11-24T06:15:52.789034+0000","last_peered":"2023-11-24T06:15:52.789034+0000","last_clean":"2023-11-24T06:15:52.789034+0000","last_became_active":"2023-11-24T06:15:51.153885+0000","last_became_peered":"2023-11-24T06:15:51.153885+0000","last_unstale":"2023-11-24T06:15:52.789034+0000","last_undegraded":"2023-11-24T06:15:52.789034+0000","last_fullsized":"2023-11-24T06:15:52.789034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:36:33.590350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793463+0000","last_change":"2023-11-24T06:15:51.809270+0000","last_active":"2023-11-24T06:15:52.793463+0000","last_peered":"2023-11-24T06:15:52.793463+0000","last_clean":"2023-11-24T06:15:52.793463+0000","last_became_active":"2023-11-24T06:15:51.808994+0000","last_became_peered":"2023-11-24T06:15:51.808994+0000","last_unstale":"2023-11-24T06:15:52.793463+0000","last_undegraded":"2023-11-24T06:15:52.793463+0000","last_fullsized":"2023-11-24T06:15:52.793463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:54.543519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185536+0000","last_change":"2023-11-24T06:15:51.166677+0000","last_active":"2023-11-24T06:15:54.185536+0000","last_peered":"2023-11-24T06:15:54.185536+0000","last_clean":"2023-11-24T06:15:54.185536+0000","last_became_active":"2023-11-24T06:15:51.164374+0000","last_became_peered":"2023-11-24T06:15:51.164374+0000","last_unstale":"2023-11-24T06:15:54.185536+0000","last_undegraded":"2023-11-24T06:15:54.185536+0000","last_fullsized":"2023-11-24T06:15:54.185536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:00:50.542922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788766+0000","last_change":"2023-11-24T06:15:51.808320+0000","last_active":"2023-11-24T06:15:52.788766+0000","last_peered":"2023-11-24T06:15:52.788766+0000","last_clean":"2023-11-24T06:15:52.788766+0000","last_became_active":"2023-11-24T06:15:51.804832+0000","last_became_peered":"2023-11-24T06:15:51.804832+0000","last_unstale":"2023-11-24T06:15:52.788766+0000","last_undegraded":"2023-11-24T06:15:52.788766+0000","last_fullsized":"2023-11-24T06:15:52.788766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:21:49.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.119685+0000","last_change":"2023-11-24T06:15:51.168020+0000","last_active":"2023-11-24T06:15:54.119685+0000","last_peered":"2023-11-24T06:15:54.119685+0000","last_clean":"2023-11-24T06:15:54.119685+0000","last_became_active":"2023-11-24T06:15:51.167664+0000","last_became_peered":"2023-11-24T06:15:51.167664+0000","last_unstale":"2023-11-24T06:15:54.119685+0000","last_undegraded":"2023-11-24T06:15:54.119685+0000","last_fullsized":"2023-11-24T06:15:54.119685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:17:00.705564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845799+0000","last_change":"2023-11-24T06:15:51.816931+0000","last_active":"2023-11-24T06:15:53.845799+0000","last_peered":"2023-11-24T06:15:53.845799+0000","last_clean":"2023-11-24T06:15:53.845799+0000","last_became_active":"2023-11-24T06:15:51.816611+0000","last_became_peered":"2023-11-24T06:15:51.816611+0000","last_unstale":"2023-11-24T06:15:53.845799+0000","last_undegraded":"2023-11-24T06:15:53.845799+0000","last_fullsized":"2023-11-24T06:15:53.845799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:27.816377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750733+0000","last_change":"2023-11-24T06:15:51.172657+0000","last_active":"2023-11-24T06:15:53.750733+0000","last_peered":"2023-11-24T06:15:53.750733+0000","last_clean":"2023-11-24T06:15:53.750733+0000","last_became_active":"2023-11-24T06:15:51.172152+0000","last_became_peered":"2023-11-24T06:15:51.172152+0000","last_unstale":"2023-11-24T06:15:53.750733+0000","last_undegraded":"2023-11-24T06:15:53.750733+0000","last_fullsized":"2023-11-24T06:15:53.750733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:23:43.714712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183046+0000","last_change":"2023-11-24T06:15:51.815689+0000","last_active":"2023-11-24T06:15:54.183046+0000","last_peered":"2023-11-24T06:15:54.183046+0000","last_clean":"2023-11-24T06:15:54.183046+0000","last_became_active":"2023-11-24T06:15:51.814742+0000","last_became_peered":"2023-11-24T06:15:51.814742+0000","last_unstale":"2023-11-24T06:15:54.183046+0000","last_undegraded":"2023-11-24T06:15:54.183046+0000","last_fullsized":"2023-11-24T06:15:54.183046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:05.782767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183694+0000","last_change":"2023-11-24T06:15:51.164574+0000","last_active":"2023-11-24T06:15:54.183694+0000","last_peered":"2023-11-24T06:15:54.183694+0000","last_clean":"2023-11-24T06:15:54.183694+0000","last_became_active":"2023-11-24T06:15:51.163261+0000","last_became_peered":"2023-11-24T06:15:51.163261+0000","last_unstale":"2023-11-24T06:15:54.183694+0000","last_undegraded":"2023-11-24T06:15:54.183694+0000","last_fullsized":"2023-11-24T06:15:54.183694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:19:30.567064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791508+0000","last_change":"2023-11-24T06:15:51.808104+0000","last_active":"2023-11-24T06:15:52.791508+0000","last_peered":"2023-11-24T06:15:52.791508+0000","last_clean":"2023-11-24T06:15:52.791508+0000","last_became_active":"2023-11-24T06:15:51.807649+0000","last_became_peered":"2023-11-24T06:15:51.807649+0000","last_unstale":"2023-11-24T06:15:52.791508+0000","last_undegraded":"2023-11-24T06:15:52.791508+0000","last_fullsized":"2023-11-24T06:15:52.791508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:47.469041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788424+0000","last_change":"2023-11-24T06:15:51.160561+0000","last_active":"2023-11-24T06:15:52.788424+0000","last_peered":"2023-11-24T06:15:52.788424+0000","last_clean":"2023-11-24T06:15:52.788424+0000","last_became_active":"2023-11-24T06:15:51.160266+0000","last_became_peered":"2023-11-24T06:15:51.160266+0000","last_unstale":"2023-11-24T06:15:52.788424+0000","last_undegraded":"2023-11-24T06:15:52.788424+0000","last_fullsized":"2023-11-24T06:15:52.788424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:09:38.300549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845022+0000","last_change":"2023-11-24T06:15:51.806779+0000","last_active":"2023-11-24T06:15:53.845022+0000","last_peered":"2023-11-24T06:15:53.845022+0000","last_clean":"2023-11-24T06:15:53.845022+0000","last_became_active":"2023-11-24T06:15:51.806429+0000","last_became_peered":"2023-11-24T06:15:51.806429+0000","last_unstale":"2023-11-24T06:15:53.845022+0000","last_undegraded":"2023-11-24T06:15:53.845022+0000","last_fullsized":"2023-11-24T06:15:53.845022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:01.976941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788974+0000","last_change":"2023-11-24T06:15:51.157354+0000","last_active":"2023-11-24T06:15:52.788974+0000","last_peered":"2023-11-24T06:15:52.788974+0000","last_clean":"2023-11-24T06:15:52.788974+0000","last_became_active":"2023-11-24T06:15:51.156981+0000","last_became_peered":"2023-11-24T06:15:51.156981+0000","last_unstale":"2023-11-24T06:15:52.788974+0000","last_undegraded":"2023-11-24T06:15:52.788974+0000","last_fullsized":"2023-11-24T06:15:52.788974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:19:30.567064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845730+0000","last_change":"2023-11-24T06:15:51.823678+0000","last_active":"2023-11-24T06:15:53.845730+0000","last_peered":"2023-11-24T06:15:53.845730+0000","last_clean":"2023-11-24T06:15:53.845730+0000","last_became_active":"2023-11-24T06:15:51.823372+0000","last_became_peered":"2023-11-24T06:15:51.823372+0000","last_unstale":"2023-11-24T06:15:53.845730+0000","last_undegraded":"2023-11-24T06:15:53.845730+0000","last_fullsized":"2023-11-24T06:15:53.845730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:46.484813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184891+0000","last_change":"2023-11-24T06:15:51.167493+0000","last_active":"2023-11-24T06:15:54.184891+0000","last_peered":"2023-11-24T06:15:54.184891+0000","last_clean":"2023-11-24T06:15:54.184891+0000","last_became_active":"2023-11-24T06:15:51.167110+0000","last_became_peered":"2023-11-24T06:15:51.167110+0000","last_unstale":"2023-11-24T06:15:54.184891+0000","last_undegraded":"2023-11-24T06:15:54.184891+0000","last_fullsized":"2023-11-24T06:15:54.184891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:59.444337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749958+0000","last_change":"2023-11-24T06:15:51.816423+0000","last_active":"2023-11-24T06:15:53.749958+0000","last_peered":"2023-11-24T06:15:53.749958+0000","last_clean":"2023-11-24T06:15:53.749958+0000","last_became_active":"2023-11-24T06:15:51.816122+0000","last_became_peered":"2023-11-24T06:15:51.816122+0000","last_unstale":"2023-11-24T06:15:53.749958+0000","last_undegraded":"2023-11-24T06:15:53.749958+0000","last_fullsized":"2023-11-24T06:15:53.749958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:50.460127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793429+0000","last_change":"2023-11-24T06:15:51.803878+0000","last_active":"2023-11-24T06:15:52.793429+0000","last_peered":"2023-11-24T06:15:52.793429+0000","last_clean":"2023-11-24T06:15:52.793429+0000","last_became_active":"2023-11-24T06:15:51.803472+0000","last_became_peered":"2023-11-24T06:15:51.803472+0000","last_unstale":"2023-11-24T06:15:52.793429+0000","last_undegraded":"2023-11-24T06:15:52.793429+0000","last_fullsized":"2023-11-24T06:15:52.793429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:23.431671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184531+0000","last_change":"2023-11-24T06:15:51.177433+0000","last_active":"2023-11-24T06:15:54.184531+0000","last_peered":"2023-11-24T06:15:54.184531+0000","last_clean":"2023-11-24T06:15:54.184531+0000","last_became_active":"2023-11-24T06:15:51.176987+0000","last_became_peered":"2023-11-24T06:15:51.176987+0000","last_unstale":"2023-11-24T06:15:54.184531+0000","last_undegraded":"2023-11-24T06:15:54.184531+0000","last_fullsized":"2023-11-24T06:15:54.184531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:18.981340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793828+0000","last_change":"2023-11-24T06:15:51.815123+0000","last_active":"2023-11-24T06:15:52.793828+0000","last_peered":"2023-11-24T06:15:52.793828+0000","last_clean":"2023-11-24T06:15:52.793828+0000","last_became_active":"2023-11-24T06:15:51.814778+0000","last_became_peered":"2023-11-24T06:15:51.814778+0000","last_unstale":"2023-11-24T06:15:52.793828+0000","last_undegraded":"2023-11-24T06:15:52.793828+0000","last_fullsized":"2023-11-24T06:15:52.793828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:12.167390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185069+0000","last_change":"2023-11-24T06:15:51.169354+0000","last_active":"2023-11-24T06:15:54.185069+0000","last_peered":"2023-11-24T06:15:54.185069+0000","last_clean":"2023-11-24T06:15:54.185069+0000","last_became_active":"2023-11-24T06:15:51.168916+0000","last_became_peered":"2023-11-24T06:15:51.168916+0000","last_unstale":"2023-11-24T06:15:54.185069+0000","last_undegraded":"2023-11-24T06:15:54.185069+0000","last_fullsized":"2023-11-24T06:15:54.185069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:15.261716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.182984+0000","last_change":"2023-11-24T06:15:51.818739+0000","last_active":"2023-11-24T06:15:54.182984+0000","last_peered":"2023-11-24T06:15:54.182984+0000","last_clean":"2023-11-24T06:15:54.182984+0000","last_became_active":"2023-11-24T06:15:51.817316+0000","last_became_peered":"2023-11-24T06:15:51.817316+0000","last_unstale":"2023-11-24T06:15:54.182984+0000","last_undegraded":"2023-11-24T06:15:54.182984+0000","last_fullsized":"2023-11-24T06:15:54.182984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:12:12.520019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845707+0000","last_change":"2023-11-24T06:15:51.152517+0000","last_active":"2023-11-24T06:15:53.845707+0000","last_peered":"2023-11-24T06:15:53.845707+0000","last_clean":"2023-11-24T06:15:53.845707+0000","last_became_active":"2023-11-24T06:15:51.152218+0000","last_became_peered":"2023-11-24T06:15:51.152218+0000","last_unstale":"2023-11-24T06:15:53.845707+0000","last_undegraded":"2023-11-24T06:15:53.845707+0000","last_fullsized":"2023-11-24T06:15:53.845707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:59.444337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845977+0000","last_change":"2023-11-24T06:15:51.817856+0000","last_active":"2023-11-24T06:15:53.845977+0000","last_peered":"2023-11-24T06:15:53.845977+0000","last_clean":"2023-11-24T06:15:53.845977+0000","last_became_active":"2023-11-24T06:15:51.817558+0000","last_became_peered":"2023-11-24T06:15:51.817558+0000","last_unstale":"2023-11-24T06:15:53.845977+0000","last_undegraded":"2023-11-24T06:15:53.845977+0000","last_fullsized":"2023-11-24T06:15:53.845977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:47.291087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.129254+0000","last_change":"2023-11-24T06:15:51.157608+0000","last_active":"2023-11-24T06:15:54.129254+0000","last_peered":"2023-11-24T06:15:54.129254+0000","last_clean":"2023-11-24T06:15:54.129254+0000","last_became_active":"2023-11-24T06:15:51.157330+0000","last_became_peered":"2023-11-24T06:15:51.157330+0000","last_unstale":"2023-11-24T06:15:54.129254+0000","last_undegraded":"2023-11-24T06:15:54.129254+0000","last_fullsized":"2023-11-24T06:15:54.129254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:39:05.215342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.637}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":78220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8220000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":79706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7410000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4890000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.673}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.161}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4820000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6880000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":78184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.573}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":233472,"data_stored":79706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":78094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":78254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":668381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:16:02.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:16:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.494+0000 7f6f228a8640 1 Processor -- start 2023-11-24T06:16:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.494+0000 7f6f228a8640 1 -- start start 2023-11-24T06:16:02.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.495+0000 7f6f228a8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f1c143950 0x7f6f1c164e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.495+0000 7f6f228a8640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1c071700 con 0x7f6f1c0cf760 2023-11-24T06:16:02.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.495+0000 7f6f228a8640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1c15b410 con 0x7f6f1c143570 2023-11-24T06:16:02.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.495+0000 7f6f228a8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c15b130 con 0x7f6f1c143950 2023-11-24T06:16:02.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.495+0000 7f6f210a5640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6f1c0cf760 0x7f6f1c15a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56604/0 (socket says 172.21.15.5:56604) 2023-11-24T06:16:02.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.495+0000 7f6f210a5640 1 -- 172.21.15.5:0/82024897 learned_addr learned my addr 172.21.15.5:0/82024897 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.496+0000 7f6f220a7640 1 --2- 172.21.15.5:0/82024897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f1c143950 0x7f6f1c164e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.496+0000 7f6f208a4640 1 -- 172.21.15.5:0/82024897 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535899934 0 0) 0x7f6f0c001110 con 0x7f6f1c0cf760 2023-11-24T06:16:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.496+0000 7f6f208a4640 1 -- 172.21.15.5:0/82024897 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f08003520 con 0x7f6f1c0cf760 2023-11-24T06:16:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.496+0000 7f6f208a4640 1 -- 172.21.15.5:0/82024897 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2260662710 0 0) 0x7f6f08003520 con 0x7f6f1c0cf760 2023-11-24T06:16:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.496+0000 7f6f208a4640 1 -- 172.21.15.5:0/82024897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f1c143950 msgr2=0x7f6f1c164e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.496+0000 7f6f208a4640 1 --2- 172.21.15.5:0/82024897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f1c143950 0x7f6f1c164e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.496+0000 7f6f208a4640 1 -- 172.21.15.5:0/82024897 >> v1:172.21.15.6:6789/0 conn(0x7f6f1c143570 legacy=0x7f6f1c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.496+0000 7f6f208a4640 1 -- 172.21.15.5:0/82024897 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c15dbd0 con 0x7f6f1c0cf760 2023-11-24T06:16:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.496+0000 7f6f208a4640 1 -- 172.21.15.5:0/82024897 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f6f0c001800 con 0x7f6f1c0cf760 2023-11-24T06:16:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.496+0000 7f6f208a4640 1 -- 172.21.15.5:0/82024897 >> v1:172.21.15.5:6789/0 conn(0x7f6f1c0cf760 legacy=0x7f6f1c15a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.496+0000 7f6f208a4640 1 --2- 172.21.15.5:0/82024897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f1c143950 0x7f6f08003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.497+0000 7f6f208a4640 1 --2- 172.21.15.5:0/82024897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f08003f90 0x7f6f080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.497+0000 7f6f208a4640 1 --2- 172.21.15.5:0/82024897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08004d00 0x7f6f08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.497+0000 7f6f208a4640 1 -- 172.21.15.5:0/82024897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c15b130 con 0x7f6f08003f90 2023-11-24T06:16:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.497+0000 7f6f208a4640 1 -- 172.21.15.5:0/82024897 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0800bb70 con 0x7f6f08004d00 2023-11-24T06:16:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.497+0000 7f6f208a4640 1 -- 172.21.15.5:0/82024897 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0800df40 con 0x7f6f1c143950 2023-11-24T06:16:02.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.497+0000 7f6f210a5640 1 --2- 172.21.15.5:0/82024897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f08003f90 0x7f6f080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.497+0000 7f6f218a6640 1 --2- 172.21.15.5:0/82024897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f1c143950 0x7f6f08003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.497+0000 7f6f220a7640 1 --2- 172.21.15.5:0/82024897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08004d00 0x7f6f08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.497+0000 7f6f210a5640 1 -- 172.21.15.5:0/82024897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f1c143950 msgr2=0x7f6f08003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:02.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.497+0000 7f6f210a5640 1 --2- 172.21.15.5:0/82024897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f1c143950 0x7f6f08003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.497+0000 7f6f210a5640 1 -- 172.21.15.5:0/82024897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08004d00 msgr2=0x7f6f08005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:02.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.497+0000 7f6f210a5640 1 --2- 172.21.15.5:0/82024897 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08004d00 0x7f6f08005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.497+0000 7f6f210a5640 1 -- 172.21.15.5:0/82024897 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f08003520 con 0x7f6f08003f90 2023-11-24T06:16:02.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.497+0000 7f6f218a6640 1 --2- 172.21.15.5:0/82024897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f1c143950 0x7f6f08003a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:02.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.498+0000 7f6f210a5640 1 --2- 172.21.15.5:0/82024897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f08003f90 0x7f6f080047c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00c210 tx=0x7f6f080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9164b08c8d754ecb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:02.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.499+0000 7f6f208a4640 1 -- 172.21.15.5:0/82024897 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c005ad0 con 0x7f6f08003f90 2023-11-24T06:16:02.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.499+0000 7f6f208a4640 1 -- 172.21.15.5:0/82024897 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c005c50 con 0x7f6f08003f90 2023-11-24T06:16:02.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.499+0000 7f6f208a4640 1 -- 172.21.15.5:0/82024897 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c02c890 con 0x7f6f08003f90 2023-11-24T06:16:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.499+0000 7f6f228a8640 1 -- 172.21.15.5:0/82024897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f08003f90 msgr2=0x7f6f080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.499+0000 7f6f228a8640 1 --2- 172.21.15.5:0/82024897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f08003f90 0x7f6f080047c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00c210 tx=0x7f6f080057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.499+0000 7f6f220a7640 1 -- 172.21.15.5:0/82024897 reap_dead start 2023-11-24T06:16:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.499+0000 7f6f228a8640 1 -- 172.21.15.5:0/82024897 shutdown_connections 2023-11-24T06:16:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.499+0000 7f6f228a8640 1 -- 172.21.15.5:0/82024897 >> 172.21.15.5:0/82024897 conn(0x7f6f1c06a860 msgr2=0x7f6f1c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.499+0000 7f6f228a8640 1 -- 172.21.15.5:0/82024897 shutdown_connections 2023-11-24T06:16:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.499+0000 7f6f228a8640 1 -- 172.21.15.5:0/82024897 wait complete. 2023-11-24T06:16:02.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.501+0000 7f6f228a8640 1 Processor -- start 2023-11-24T06:16:02.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.501+0000 7f6f228a8640 1 -- start start 2023-11-24T06:16:02.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.502+0000 7f6f228a8640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08003f90 0x7f6f1c159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.502+0000 7f6f228a8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f1c13b4f0 0x7f6f1c139910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.502+0000 7f6f228a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f1c0cabb0 0x7f6f1c0cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.502+0000 7f6f228a8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0cb570 con 0x7f6f1c0cabb0 2023-11-24T06:16:02.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.502+0000 7f6f228a8640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0c3b20 con 0x7f6f08003f90 2023-11-24T06:16:02.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.502+0000 7f6f228a8640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c13c650 con 0x7f6f1c13b4f0 2023-11-24T06:16:02.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.502+0000 7f6f220a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f1c0cabb0 0x7f6f1c0cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.502+0000 7f6f220a7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f1c0cabb0 0x7f6f1c0cb030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36064/0 (socket says 172.21.15.5:36064) 2023-11-24T06:16:02.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.502+0000 7f6f220a7640 1 -- 172.21.15.5:0/1986962512 learned_addr learned my addr 172.21.15.5:0/1986962512 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:02.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.503+0000 7f6f220a7640 1 -- 172.21.15.5:0/1986962512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f1c13b4f0 msgr2=0x7f6f1c139910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.503+0000 7f6f220a7640 1 --2- 172.21.15.5:0/1986962512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f1c13b4f0 0x7f6f1c139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.503+0000 7f6f220a7640 1 -- 172.21.15.5:0/1986962512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08003f90 msgr2=0x7f6f1c159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.503+0000 7f6f220a7640 1 --2- 172.21.15.5:0/1986962512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08003f90 0x7f6f1c159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.503+0000 7f6f220a7640 1 -- 172.21.15.5:0/1986962512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c13ba40 con 0x7f6f1c0cabb0 2023-11-24T06:16:02.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.503+0000 7f6f220a7640 1 --2- 172.21.15.5:0/1986962512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f1c0cabb0 0x7f6f1c0cb030 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c06e310 tx=0x7f6f1400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:02.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.503+0000 7f6f127fc640 1 -- 172.21.15.5:0/1986962512 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f14017400 con 0x7f6f1c0cabb0 2023-11-24T06:16:02.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.503+0000 7f6f127fc640 1 -- 172.21.15.5:0/1986962512 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f14011040 con 0x7f6f1c0cabb0 2023-11-24T06:16:02.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.503+0000 7f6f127fc640 1 -- 172.21.15.5:0/1986962512 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f140177c0 con 0x7f6f1c0cabb0 2023-11-24T06:16:02.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.503+0000 7f6f228a8640 1 -- 172.21.15.5:0/1986962512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1c13bd20 con 0x7f6f1c0cabb0 2023-11-24T06:16:02.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.503+0000 7f6f228a8640 1 -- 172.21.15.5:0/1986962512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1c13c2e0 con 0x7f6f1c0cabb0 2023-11-24T06:16:02.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.505+0000 7f6f228a8640 1 -- 172.21.15.5:0/1986962512 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1c071b10 con 0x7f6f1c0cabb0 2023-11-24T06:16:02.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.505+0000 7f6f127fc640 1 -- 172.21.15.5:0/1986962512 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6f14017920 con 0x7f6f1c0cabb0 2023-11-24T06:16:02.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.506+0000 7f6f127fc640 1 --2- 172.21.15.5:0/1986962512 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6f0006ae40 0x7f6f0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.506+0000 7f6f127fc640 1 -- 172.21.15.5:0/1986962512 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f14089c10 con 0x7f6f1c0cabb0 2023-11-24T06:16:02.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.506+0000 7f6f218a6640 1 --2- 172.21.15.5:0/1986962512 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6f0006ae40 0x7f6f0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.507+0000 7f6f218a6640 1 --2- 172.21.15.5:0/1986962512 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6f0006ae40 0x7f6f0006d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6f180016d0 tx=0x7f6f18009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:02.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.511+0000 7f6f127fc640 1 -- 172.21.15.5:0/1986962512 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f14054b90 con 0x7f6f1c0cabb0 2023-11-24T06:16:02.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.691+0000 7f6f228a8640 1 -- 172.21.15.5:0/1986962512 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6f1c15ad40 con 0x7f6f0006ae40 2023-11-24T06:16:02.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.699+0000 7f6f127fc640 1 -- 172.21.15.5:0/1986962512 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264488 (secure 0 0 0) 0x7f6f1c15ad40 con 0x7f6f0006ae40 2023-11-24T06:16:02.706 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:02.707 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-11-24T06:16:02.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.705+0000 7f6f228a8640 1 -- 172.21.15.5:0/1986962512 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6f0006ae40 msgr2=0x7f6f0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.705+0000 7f6f228a8640 1 --2- 172.21.15.5:0/1986962512 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6f0006ae40 0x7f6f0006d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6f180016d0 tx=0x7f6f18009040 comp rx=0 tx=0).stop 2023-11-24T06:16:02.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.705+0000 7f6f228a8640 1 -- 172.21.15.5:0/1986962512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f1c0cabb0 msgr2=0x7f6f1c0cb030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.705+0000 7f6f228a8640 1 --2- 172.21.15.5:0/1986962512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f1c0cabb0 0x7f6f1c0cb030 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c06e310 tx=0x7f6f1400f4a0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.705+0000 7f6f228a8640 1 -- 172.21.15.5:0/1986962512 shutdown_connections 2023-11-24T06:16:02.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.705+0000 7f6f228a8640 1 --2- 172.21.15.5:0/1986962512 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6f0006ae40 0x7f6f0006d300 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.705+0000 7f6f228a8640 1 --2- 172.21.15.5:0/1986962512 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6f1c13b4f0 0x7f6f1c139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.705+0000 7f6f228a8640 1 --2- 172.21.15.5:0/1986962512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f1c0cabb0 0x7f6f1c0cb030 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.705+0000 7f6f228a8640 1 --2- 172.21.15.5:0/1986962512 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6f08003f90 0x7f6f1c159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.705+0000 7f6f228a8640 1 -- 172.21.15.5:0/1986962512 >> 172.21.15.5:0/1986962512 conn(0x7f6f1c06a860 msgr2=0x7f6f1c0ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.705+0000 7f6f228a8640 1 -- 172.21.15.5:0/1986962512 shutdown_connections 2023-11-24T06:16:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.705+0000 7f6f228a8640 1 -- 172.21.15.5:0/1986962512 wait complete. 2023-11-24T06:16:02.725 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-24T06:16:02.090772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235764,"kb_used_data":2948,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738956,"statfs":{"total":773094113280,"available":772852690944,"internally_reserved":0,"allocated":3018752,"data_stored":1809589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003830"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185145+0000","last_change":"2023-11-24T06:15:51.826913+0000","last_active":"2023-11-24T06:15:54.185145+0000","last_peered":"2023-11-24T06:15:54.185145+0000","last_clean":"2023-11-24T06:15:54.185145+0000","last_became_active":"2023-11-24T06:15:51.826683+0000","last_became_peered":"2023-11-24T06:15:51.826683+0000","last_unstale":"2023-11-24T06:15:54.185145+0000","last_undegraded":"2023-11-24T06:15:54.185145+0000","last_fullsized":"2023-11-24T06:15:54.185145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:45:59.277130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789469+0000","last_change":"2023-11-24T06:15:51.820105+0000","last_active":"2023-11-24T06:15:52.789469+0000","last_peered":"2023-11-24T06:15:52.789469+0000","last_clean":"2023-11-24T06:15:52.789469+0000","last_became_active":"2023-11-24T06:15:51.819862+0000","last_became_peered":"2023-11-24T06:15:51.819862+0000","last_unstale":"2023-11-24T06:15:52.789469+0000","last_undegraded":"2023-11-24T06:15:52.789469+0000","last_fullsized":"2023-11-24T06:15:52.789469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:58.501546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.182835+0000","last_change":"2023-11-24T06:15:51.817345+0000","last_active":"2023-11-24T06:15:54.182835+0000","last_peered":"2023-11-24T06:15:54.182835+0000","last_clean":"2023-11-24T06:15:54.182835+0000","last_became_active":"2023-11-24T06:15:51.815455+0000","last_became_peered":"2023-11-24T06:15:51.815455+0000","last_unstale":"2023-11-24T06:15:54.182835+0000","last_undegraded":"2023-11-24T06:15:54.182835+0000","last_fullsized":"2023-11-24T06:15:54.182835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:50.460127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793545+0000","last_change":"2023-11-24T06:15:51.811576+0000","last_active":"2023-11-24T06:15:52.793545+0000","last_peered":"2023-11-24T06:15:52.793545+0000","last_clean":"2023-11-24T06:15:52.793545+0000","last_became_active":"2023-11-24T06:15:51.811306+0000","last_became_peered":"2023-11-24T06:15:51.811306+0000","last_unstale":"2023-11-24T06:15:52.793545+0000","last_undegraded":"2023-11-24T06:15:52.793545+0000","last_fullsized":"2023-11-24T06:15:52.793545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:34.521019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184624+0000","last_change":"2023-11-24T06:15:51.821890+0000","last_active":"2023-11-24T06:15:54.184624+0000","last_peered":"2023-11-24T06:15:54.184624+0000","last_clean":"2023-11-24T06:15:54.184624+0000","last_became_active":"2023-11-24T06:15:51.821632+0000","last_became_peered":"2023-11-24T06:15:51.821632+0000","last_unstale":"2023-11-24T06:15:54.184624+0000","last_undegraded":"2023-11-24T06:15:54.184624+0000","last_fullsized":"2023-11-24T06:15:54.184624+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:54.384409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792886+0000","last_change":"2023-11-24T06:15:51.828326+0000","last_active":"2023-11-24T06:15:52.792886+0000","last_peered":"2023-11-24T06:15:52.792886+0000","last_clean":"2023-11-24T06:15:52.792886+0000","last_became_active":"2023-11-24T06:15:51.827922+0000","last_became_peered":"2023-11-24T06:15:51.827922+0000","last_unstale":"2023-11-24T06:15:52.792886+0000","last_undegraded":"2023-11-24T06:15:52.792886+0000","last_fullsized":"2023-11-24T06:15:52.792886+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:25.825619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183858+0000","last_change":"2023-11-24T06:15:51.825625+0000","last_active":"2023-11-24T06:15:54.183858+0000","last_peered":"2023-11-24T06:15:54.183858+0000","last_clean":"2023-11-24T06:15:54.183858+0000","last_became_active":"2023-11-24T06:15:51.825308+0000","last_became_peered":"2023-11-24T06:15:51.825308+0000","last_unstale":"2023-11-24T06:15:54.183858+0000","last_undegraded":"2023-11-24T06:15:54.183858+0000","last_fullsized":"2023-11-24T06:15:54.183858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:58.501546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844647+0000","last_change":"2023-11-24T06:15:51.817175+0000","last_active":"2023-11-24T06:15:53.844647+0000","last_peered":"2023-11-24T06:15:53.844647+0000","last_clean":"2023-11-24T06:15:53.844647+0000","last_became_active":"2023-11-24T06:15:51.816363+0000","last_became_peered":"2023-11-24T06:15:51.816363+0000","last_unstale":"2023-11-24T06:15:53.844647+0000","last_undegraded":"2023-11-24T06:15:53.844647+0000","last_fullsized":"2023-11-24T06:15:53.844647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:00:51.554978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185142+0000","last_change":"2023-11-24T06:15:51.829604+0000","last_active":"2023-11-24T06:15:54.185142+0000","last_peered":"2023-11-24T06:15:54.185142+0000","last_clean":"2023-11-24T06:15:54.185142+0000","last_became_active":"2023-11-24T06:15:51.829291+0000","last_became_peered":"2023-11-24T06:15:51.829291+0000","last_unstale":"2023-11-24T06:15:54.185142+0000","last_undegraded":"2023-11-24T06:15:54.185142+0000","last_fullsized":"2023-11-24T06:15:54.185142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:57:53.020152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750691+0000","last_change":"2023-11-24T06:15:51.820996+0000","last_active":"2023-11-24T06:15:53.750691+0000","last_peered":"2023-11-24T06:15:53.750691+0000","last_clean":"2023-11-24T06:15:53.750691+0000","last_became_active":"2023-11-24T06:15:51.820018+0000","last_became_peered":"2023-11-24T06:15:51.820018+0000","last_unstale":"2023-11-24T06:15:53.750691+0000","last_undegraded":"2023-11-24T06:15:53.750691+0000","last_fullsized":"2023-11-24T06:15:53.750691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:05.782767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750915+0000","last_change":"2023-11-24T06:15:51.806762+0000","last_active":"2023-11-24T06:15:53.750915+0000","last_peered":"2023-11-24T06:15:53.750915+0000","last_clean":"2023-11-24T06:15:53.750915+0000","last_became_active":"2023-11-24T06:15:51.806476+0000","last_became_peered":"2023-11-24T06:15:51.806476+0000","last_unstale":"2023-11-24T06:15:53.750915+0000","last_undegraded":"2023-11-24T06:15:53.750915+0000","last_fullsized":"2023-11-24T06:15:53.750915+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:25.825619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791243+0000","last_change":"2023-11-24T06:15:51.807959+0000","last_active":"2023-11-24T06:15:52.791243+0000","last_peered":"2023-11-24T06:15:52.791243+0000","last_clean":"2023-11-24T06:15:52.791243+0000","last_became_active":"2023-11-24T06:15:51.807625+0000","last_became_peered":"2023-11-24T06:15:51.807625+0000","last_unstale":"2023-11-24T06:15:52.791243+0000","last_undegraded":"2023-11-24T06:15:52.791243+0000","last_fullsized":"2023-11-24T06:15:52.791243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:37.913398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845276+0000","last_change":"2023-11-24T06:15:51.820393+0000","last_active":"2023-11-24T06:15:53.845276+0000","last_peered":"2023-11-24T06:15:53.845276+0000","last_clean":"2023-11-24T06:15:53.845276+0000","last_became_active":"2023-11-24T06:15:51.820092+0000","last_became_peered":"2023-11-24T06:15:51.820092+0000","last_unstale":"2023-11-24T06:15:53.845276+0000","last_undegraded":"2023-11-24T06:15:53.845276+0000","last_fullsized":"2023-11-24T06:15:53.845276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:16.273773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.182517+0000","last_change":"2023-11-24T06:15:51.822418+0000","last_active":"2023-11-24T06:15:54.182517+0000","last_peered":"2023-11-24T06:15:54.182517+0000","last_clean":"2023-11-24T06:15:54.182517+0000","last_became_active":"2023-11-24T06:15:51.821162+0000","last_became_peered":"2023-11-24T06:15:51.821162+0000","last_unstale":"2023-11-24T06:15:54.182517+0000","last_undegraded":"2023-11-24T06:15:54.182517+0000","last_fullsized":"2023-11-24T06:15:54.182517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:23:44.726769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788580+0000","last_change":"2023-11-24T06:15:51.808930+0000","last_active":"2023-11-24T06:15:52.788580+0000","last_peered":"2023-11-24T06:15:52.788580+0000","last_clean":"2023-11-24T06:15:52.788580+0000","last_became_active":"2023-11-24T06:15:51.808593+0000","last_became_peered":"2023-11-24T06:15:51.808593+0000","last_unstale":"2023-11-24T06:15:52.788580+0000","last_undegraded":"2023-11-24T06:15:52.788580+0000","last_fullsized":"2023-11-24T06:15:52.788580+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:43.291318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792334+0000","last_change":"2023-11-24T06:15:51.829868+0000","last_active":"2023-11-24T06:15:52.792334+0000","last_peered":"2023-11-24T06:15:52.792334+0000","last_clean":"2023-11-24T06:15:52.792334+0000","last_became_active":"2023-11-24T06:15:51.828875+0000","last_became_peered":"2023-11-24T06:15:51.828875+0000","last_unstale":"2023-11-24T06:15:52.792334+0000","last_undegraded":"2023-11-24T06:15:52.792334+0000","last_fullsized":"2023-11-24T06:15:52.792334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:32.225953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.794592+0000","last_change":"2023-11-24T06:15:51.814343+0000","last_active":"2023-11-24T06:15:52.794592+0000","last_peered":"2023-11-24T06:15:52.794592+0000","last_clean":"2023-11-24T06:15:52.794592+0000","last_became_active":"2023-11-24T06:15:51.814003+0000","last_became_peered":"2023-11-24T06:15:51.814003+0000","last_unstale":"2023-11-24T06:15:52.794592+0000","last_undegraded":"2023-11-24T06:15:52.794592+0000","last_fullsized":"2023-11-24T06:15:52.794592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:26:09.975671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.139925+0000","last_change":"2023-11-24T06:15:51.815884+0000","last_active":"2023-11-24T06:15:53.139925+0000","last_peered":"2023-11-24T06:15:53.139925+0000","last_clean":"2023-11-24T06:15:53.139925+0000","last_became_active":"2023-11-24T06:15:51.814549+0000","last_became_peered":"2023-11-24T06:15:51.814549+0000","last_unstale":"2023-11-24T06:15:53.139925+0000","last_undegraded":"2023-11-24T06:15:53.139925+0000","last_fullsized":"2023-11-24T06:15:53.139925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:43.354531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185458+0000","last_change":"2023-11-24T06:15:51.823265+0000","last_active":"2023-11-24T06:15:54.185458+0000","last_peered":"2023-11-24T06:15:54.185458+0000","last_clean":"2023-11-24T06:15:54.185458+0000","last_became_active":"2023-11-24T06:15:51.823026+0000","last_became_peered":"2023-11-24T06:15:51.823026+0000","last_unstale":"2023-11-24T06:15:54.185458+0000","last_undegraded":"2023-11-24T06:15:54.185458+0000","last_fullsized":"2023-11-24T06:15:54.185458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:48.210729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750151+0000","last_change":"2023-11-24T06:15:51.818614+0000","last_active":"2023-11-24T06:15:53.750151+0000","last_peered":"2023-11-24T06:15:53.750151+0000","last_clean":"2023-11-24T06:15:53.750151+0000","last_became_active":"2023-11-24T06:15:51.818250+0000","last_became_peered":"2023-11-24T06:15:51.818250+0000","last_unstale":"2023-11-24T06:15:53.750151+0000","last_undegraded":"2023-11-24T06:15:53.750151+0000","last_fullsized":"2023-11-24T06:15:53.750151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:07.460859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.182802+0000","last_change":"2023-11-24T06:15:51.815798+0000","last_active":"2023-11-24T06:15:54.182802+0000","last_peered":"2023-11-24T06:15:54.182802+0000","last_clean":"2023-11-24T06:15:54.182802+0000","last_became_active":"2023-11-24T06:15:51.815531+0000","last_became_peered":"2023-11-24T06:15:51.815531+0000","last_unstale":"2023-11-24T06:15:54.182802+0000","last_undegraded":"2023-11-24T06:15:54.182802+0000","last_fullsized":"2023-11-24T06:15:54.182802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:55.713409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.794291+0000","last_change":"2023-11-24T06:15:51.813391+0000","last_active":"2023-11-24T06:15:52.794291+0000","last_peered":"2023-11-24T06:15:52.794291+0000","last_clean":"2023-11-24T06:15:52.794291+0000","last_became_active":"2023-11-24T06:15:51.813066+0000","last_became_peered":"2023-11-24T06:15:51.813066+0000","last_unstale":"2023-11-24T06:15:52.794291+0000","last_undegraded":"2023-11-24T06:15:52.794291+0000","last_fullsized":"2023-11-24T06:15:52.794291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:54:56.970614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844721+0000","last_change":"2023-11-24T06:15:51.812820+0000","last_active":"2023-11-24T06:15:53.844721+0000","last_peered":"2023-11-24T06:15:53.844721+0000","last_clean":"2023-11-24T06:15:53.844721+0000","last_became_active":"2023-11-24T06:15:51.812517+0000","last_became_peered":"2023-11-24T06:15:51.812517+0000","last_unstale":"2023-11-24T06:15:53.844721+0000","last_undegraded":"2023-11-24T06:15:53.844721+0000","last_fullsized":"2023-11-24T06:15:53.844721+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:25.102989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792262+0000","last_change":"2023-11-24T06:15:51.808797+0000","last_active":"2023-11-24T06:15:52.792262+0000","last_peered":"2023-11-24T06:15:52.792262+0000","last_clean":"2023-11-24T06:15:52.792262+0000","last_became_active":"2023-11-24T06:15:51.808481+0000","last_became_peered":"2023-11-24T06:15:51.808481+0000","last_unstale":"2023-11-24T06:15:52.792262+0000","last_undegraded":"2023-11-24T06:15:52.792262+0000","last_fullsized":"2023-11-24T06:15:52.792262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:05.782767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845642+0000","last_change":"2023-11-24T06:15:51.818331+0000","last_active":"2023-11-24T06:15:53.845642+0000","last_peered":"2023-11-24T06:15:53.845642+0000","last_clean":"2023-11-24T06:15:53.845642+0000","last_became_active":"2023-11-24T06:15:51.818011+0000","last_became_peered":"2023-11-24T06:15:51.818011+0000","last_unstale":"2023-11-24T06:15:53.845642+0000","last_undegraded":"2023-11-24T06:15:53.845642+0000","last_fullsized":"2023-11-24T06:15:53.845642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:58.501546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750817+0000","last_change":"2023-11-24T06:15:51.819933+0000","last_active":"2023-11-24T06:15:53.750817+0000","last_peered":"2023-11-24T06:15:53.750817+0000","last_clean":"2023-11-24T06:15:53.750817+0000","last_became_active":"2023-11-24T06:15:51.819592+0000","last_became_peered":"2023-11-24T06:15:51.819592+0000","last_unstale":"2023-11-24T06:15:53.750817+0000","last_undegraded":"2023-11-24T06:15:53.750817+0000","last_fullsized":"2023-11-24T06:15:53.750817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:32.225953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788974+0000","last_change":"2023-11-24T06:15:51.810396+0000","last_active":"2023-11-24T06:15:52.788974+0000","last_peered":"2023-11-24T06:15:52.788974+0000","last_clean":"2023-11-24T06:15:52.788974+0000","last_became_active":"2023-11-24T06:15:51.809939+0000","last_became_peered":"2023-11-24T06:15:51.809939+0000","last_unstale":"2023-11-24T06:15:52.788974+0000","last_undegraded":"2023-11-24T06:15:52.788974+0000","last_fullsized":"2023-11-24T06:15:52.788974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:16.273773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845858+0000","last_change":"2023-11-24T06:15:51.818877+0000","last_active":"2023-11-24T06:15:53.845858+0000","last_peered":"2023-11-24T06:15:53.845858+0000","last_clean":"2023-11-24T06:15:53.845858+0000","last_became_active":"2023-11-24T06:15:51.818068+0000","last_became_peered":"2023-11-24T06:15:51.818068+0000","last_unstale":"2023-11-24T06:15:53.845858+0000","last_undegraded":"2023-11-24T06:15:53.845858+0000","last_fullsized":"2023-11-24T06:15:53.845858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:34:54.384409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.182872+0000","last_change":"2023-11-24T06:15:51.821479+0000","last_active":"2023-11-24T06:15:54.182872+0000","last_peered":"2023-11-24T06:15:54.182872+0000","last_clean":"2023-11-24T06:15:54.182872+0000","last_became_active":"2023-11-24T06:15:51.820978+0000","last_became_peered":"2023-11-24T06:15:51.820978+0000","last_unstale":"2023-11-24T06:15:54.182872+0000","last_undegraded":"2023-11-24T06:15:54.182872+0000","last_fullsized":"2023-11-24T06:15:54.182872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:57:57.197250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793272+0000","last_change":"2023-11-24T06:15:51.808157+0000","last_active":"2023-11-24T06:15:52.793272+0000","last_peered":"2023-11-24T06:15:52.793272+0000","last_clean":"2023-11-24T06:15:52.793272+0000","last_became_active":"2023-11-24T06:15:51.807868+0000","last_became_peered":"2023-11-24T06:15:51.807868+0000","last_unstale":"2023-11-24T06:15:52.793272+0000","last_undegraded":"2023-11-24T06:15:52.793272+0000","last_fullsized":"2023-11-24T06:15:52.793272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:41:05.120915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.140134+0000","last_change":"2023-11-24T06:15:51.809534+0000","last_active":"2023-11-24T06:15:53.140134+0000","last_peered":"2023-11-24T06:15:53.140134+0000","last_clean":"2023-11-24T06:15:53.140134+0000","last_became_active":"2023-11-24T06:15:51.809037+0000","last_became_peered":"2023-11-24T06:15:51.809037+0000","last_unstale":"2023-11-24T06:15:53.140134+0000","last_undegraded":"2023-11-24T06:15:53.140134+0000","last_fullsized":"2023-11-24T06:15:53.140134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:38.032032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750282+0000","last_change":"2023-11-24T06:15:51.825619+0000","last_active":"2023-11-24T06:15:53.750282+0000","last_peered":"2023-11-24T06:15:53.750282+0000","last_clean":"2023-11-24T06:15:53.750282+0000","last_became_active":"2023-11-24T06:15:51.825149+0000","last_became_peered":"2023-11-24T06:15:51.825149+0000","last_unstale":"2023-11-24T06:15:53.750282+0000","last_undegraded":"2023-11-24T06:15:53.750282+0000","last_fullsized":"2023-11-24T06:15:53.750282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:12:12.520019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791884+0000","last_change":"2023-11-24T06:15:51.810514+0000","last_active":"2023-11-24T06:15:52.791884+0000","last_peered":"2023-11-24T06:15:52.791884+0000","last_clean":"2023-11-24T06:15:52.791884+0000","last_became_active":"2023-11-24T06:15:51.807943+0000","last_became_peered":"2023-11-24T06:15:51.807943+0000","last_unstale":"2023-11-24T06:15:52.791884+0000","last_undegraded":"2023-11-24T06:15:52.791884+0000","last_fullsized":"2023-11-24T06:15:52.791884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:19.993397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845626+0000","last_change":"2023-11-24T06:15:51.164865+0000","last_active":"2023-11-24T06:15:53.845626+0000","last_peered":"2023-11-24T06:15:53.845626+0000","last_clean":"2023-11-24T06:15:53.845626+0000","last_became_active":"2023-11-24T06:15:51.164524+0000","last_became_peered":"2023-11-24T06:15:51.164524+0000","last_unstale":"2023-11-24T06:15:53.845626+0000","last_undegraded":"2023-11-24T06:15:53.845626+0000","last_fullsized":"2023-11-24T06:15:53.845626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:02.689172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183027+0000","last_change":"2023-11-24T06:15:51.818737+0000","last_active":"2023-11-24T06:15:54.183027+0000","last_peered":"2023-11-24T06:15:54.183027+0000","last_clean":"2023-11-24T06:15:54.183027+0000","last_became_active":"2023-11-24T06:15:51.817354+0000","last_became_peered":"2023-11-24T06:15:51.817354+0000","last_unstale":"2023-11-24T06:15:54.183027+0000","last_undegraded":"2023-11-24T06:15:54.183027+0000","last_fullsized":"2023-11-24T06:15:54.183027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:07.460859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749981+0000","last_change":"2023-11-24T06:15:51.169166+0000","last_active":"2023-11-24T06:15:53.749981+0000","last_peered":"2023-11-24T06:15:53.749981+0000","last_clean":"2023-11-24T06:15:53.749981+0000","last_became_active":"2023-11-24T06:15:51.168863+0000","last_became_peered":"2023-11-24T06:15:51.168863+0000","last_unstale":"2023-11-24T06:15:53.749981+0000","last_undegraded":"2023-11-24T06:15:53.749981+0000","last_fullsized":"2023-11-24T06:15:53.749981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:16.146869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844773+0000","last_change":"2023-11-24T06:15:51.165957+0000","last_active":"2023-11-24T06:15:53.844773+0000","last_peered":"2023-11-24T06:15:53.844773+0000","last_clean":"2023-11-24T06:15:53.844773+0000","last_became_active":"2023-11-24T06:15:51.165575+0000","last_became_peered":"2023-11-24T06:15:51.165575+0000","last_unstale":"2023-11-24T06:15:53.844773+0000","last_undegraded":"2023-11-24T06:15:53.844773+0000","last_fullsized":"2023-11-24T06:15:53.844773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:09.461899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.844747+0000","last_change":"2023-11-24T06:15:51.818508+0000","last_active":"2023-11-24T06:15:53.844747+0000","last_peered":"2023-11-24T06:15:53.844747+0000","last_clean":"2023-11-24T06:15:53.844747+0000","last_became_active":"2023-11-24T06:15:51.818158+0000","last_became_peered":"2023-11-24T06:15:51.818158+0000","last_unstale":"2023-11-24T06:15:53.844747+0000","last_undegraded":"2023-11-24T06:15:53.844747+0000","last_fullsized":"2023-11-24T06:15:53.844747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:43.291318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792534+0000","last_change":"2023-11-24T06:15:51.168437+0000","last_active":"2023-11-24T06:15:52.792534+0000","last_peered":"2023-11-24T06:15:52.792534+0000","last_clean":"2023-11-24T06:15:52.792534+0000","last_became_active":"2023-11-24T06:15:51.162079+0000","last_became_peered":"2023-11-24T06:15:51.162079+0000","last_unstale":"2023-11-24T06:15:52.792534+0000","last_undegraded":"2023-11-24T06:15:52.792534+0000","last_fullsized":"2023-11-24T06:15:52.792534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:07.834230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788318+0000","last_change":"2023-11-24T06:15:51.811023+0000","last_active":"2023-11-24T06:15:52.788318+0000","last_peered":"2023-11-24T06:15:52.788318+0000","last_clean":"2023-11-24T06:15:52.788318+0000","last_became_active":"2023-11-24T06:15:51.810368+0000","last_became_peered":"2023-11-24T06:15:51.810368+0000","last_unstale":"2023-11-24T06:15:52.788318+0000","last_undegraded":"2023-11-24T06:15:52.788318+0000","last_fullsized":"2023-11-24T06:15:52.788318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:03:25.102989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750531+0000","last_change":"2023-11-24T06:15:51.175246+0000","last_active":"2023-11-24T06:15:53.750531+0000","last_peered":"2023-11-24T06:15:53.750531+0000","last_clean":"2023-11-24T06:15:53.750531+0000","last_became_active":"2023-11-24T06:15:51.174941+0000","last_became_peered":"2023-11-24T06:15:51.174941+0000","last_unstale":"2023-11-24T06:15:53.750531+0000","last_undegraded":"2023-11-24T06:15:53.750531+0000","last_fullsized":"2023-11-24T06:15:53.750531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:48.317425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793290+0000","last_change":"2023-11-24T06:15:51.822074+0000","last_active":"2023-11-24T06:15:52.793290+0000","last_peered":"2023-11-24T06:15:52.793290+0000","last_clean":"2023-11-24T06:15:52.793290+0000","last_became_active":"2023-11-24T06:15:51.821749+0000","last_became_peered":"2023-11-24T06:15:51.821749+0000","last_unstale":"2023-11-24T06:15:52.793290+0000","last_undegraded":"2023-11-24T06:15:52.793290+0000","last_fullsized":"2023-11-24T06:15:52.793290+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:47:30.989438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793436+0000","last_change":"2023-11-24T06:15:51.175128+0000","last_active":"2023-11-24T06:15:52.793436+0000","last_peered":"2023-11-24T06:15:52.793436+0000","last_clean":"2023-11-24T06:15:52.793436+0000","last_became_active":"2023-11-24T06:15:51.173422+0000","last_became_peered":"2023-11-24T06:15:51.173422+0000","last_unstale":"2023-11-24T06:15:52.793436+0000","last_undegraded":"2023-11-24T06:15:52.793436+0000","last_fullsized":"2023-11-24T06:15:52.793436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:12:11.507962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183145+0000","last_change":"2023-11-24T06:15:51.814210+0000","last_active":"2023-11-24T06:15:54.183145+0000","last_peered":"2023-11-24T06:15:54.183145+0000","last_clean":"2023-11-24T06:15:54.183145+0000","last_became_active":"2023-11-24T06:15:51.813951+0000","last_became_peered":"2023-11-24T06:15:51.813951+0000","last_unstale":"2023-11-24T06:15:54.183145+0000","last_undegraded":"2023-11-24T06:15:54.183145+0000","last_fullsized":"2023-11-24T06:15:54.183145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:32.225953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789082+0000","last_change":"2023-11-24T06:15:51.176266+0000","last_active":"2023-11-24T06:15:52.789082+0000","last_peered":"2023-11-24T06:15:52.789082+0000","last_clean":"2023-11-24T06:15:52.789082+0000","last_became_active":"2023-11-24T06:15:51.175918+0000","last_became_peered":"2023-11-24T06:15:51.175918+0000","last_unstale":"2023-11-24T06:15:52.789082+0000","last_undegraded":"2023-11-24T06:15:52.789082+0000","last_fullsized":"2023-11-24T06:15:52.789082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:18.981340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791650+0000","last_change":"2023-11-24T06:15:51.819556+0000","last_active":"2023-11-24T06:15:52.791650+0000","last_peered":"2023-11-24T06:15:52.791650+0000","last_clean":"2023-11-24T06:15:52.791650+0000","last_became_active":"2023-11-24T06:15:51.818398+0000","last_became_peered":"2023-11-24T06:15:51.818398+0000","last_unstale":"2023-11-24T06:15:52.791650+0000","last_undegraded":"2023-11-24T06:15:52.791650+0000","last_fullsized":"2023-11-24T06:15:52.791650+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:41.065020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750080+0000","last_change":"2023-11-24T06:15:51.169595+0000","last_active":"2023-11-24T06:15:53.750080+0000","last_peered":"2023-11-24T06:15:53.750080+0000","last_clean":"2023-11-24T06:15:53.750080+0000","last_became_active":"2023-11-24T06:15:51.167755+0000","last_became_peered":"2023-11-24T06:15:51.167755+0000","last_unstale":"2023-11-24T06:15:53.750080+0000","last_undegraded":"2023-11-24T06:15:53.750080+0000","last_fullsized":"2023-11-24T06:15:53.750080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:58:37.019975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184774+0000","last_change":"2023-11-24T06:15:51.821305+0000","last_active":"2023-11-24T06:15:54.184774+0000","last_peered":"2023-11-24T06:15:54.184774+0000","last_clean":"2023-11-24T06:15:54.184774+0000","last_became_active":"2023-11-24T06:15:51.821061+0000","last_became_peered":"2023-11-24T06:15:51.821061+0000","last_unstale":"2023-11-24T06:15:54.184774+0000","last_undegraded":"2023-11-24T06:15:54.184774+0000","last_fullsized":"2023-11-24T06:15:54.184774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:24.298719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.129081+0000","last_change":"2023-11-24T06:15:51.150730+0000","last_active":"2023-11-24T06:15:54.129081+0000","last_peered":"2023-11-24T06:15:54.129081+0000","last_clean":"2023-11-24T06:15:54.129081+0000","last_became_active":"2023-11-24T06:15:51.147375+0000","last_became_peered":"2023-11-24T06:15:51.147375+0000","last_unstale":"2023-11-24T06:15:54.129081+0000","last_undegraded":"2023-11-24T06:15:54.129081+0000","last_fullsized":"2023-11-24T06:15:54.129081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:24:38.346475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.182531+0000","last_change":"2023-11-24T06:15:51.822499+0000","last_active":"2023-11-24T06:15:54.182531+0000","last_peered":"2023-11-24T06:15:54.182531+0000","last_clean":"2023-11-24T06:15:54.182531+0000","last_became_active":"2023-11-24T06:15:51.822192+0000","last_became_peered":"2023-11-24T06:15:51.822192+0000","last_unstale":"2023-11-24T06:15:54.182531+0000","last_undegraded":"2023-11-24T06:15:54.182531+0000","last_fullsized":"2023-11-24T06:15:54.182531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:49.329482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793561+0000","last_change":"2023-11-24T06:15:51.174234+0000","last_active":"2023-11-24T06:15:52.793561+0000","last_peered":"2023-11-24T06:15:52.793561+0000","last_clean":"2023-11-24T06:15:52.793561+0000","last_became_active":"2023-11-24T06:15:51.173741+0000","last_became_peered":"2023-11-24T06:15:51.173741+0000","last_unstale":"2023-11-24T06:15:52.793561+0000","last_undegraded":"2023-11-24T06:15:52.793561+0000","last_fullsized":"2023-11-24T06:15:52.793561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:16.146869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184654+0000","last_change":"2023-11-24T06:15:51.828660+0000","last_active":"2023-11-24T06:15:54.184654+0000","last_peered":"2023-11-24T06:15:54.184654+0000","last_clean":"2023-11-24T06:15:54.184654+0000","last_became_active":"2023-11-24T06:15:51.828176+0000","last_became_peered":"2023-11-24T06:15:51.828176+0000","last_unstale":"2023-11-24T06:15:54.184654+0000","last_undegraded":"2023-11-24T06:15:54.184654+0000","last_fullsized":"2023-11-24T06:15:54.184654+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:47.291087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749753+0000","last_change":"2023-11-24T06:15:51.171268+0000","last_active":"2023-11-24T06:15:53.749753+0000","last_peered":"2023-11-24T06:15:53.749753+0000","last_clean":"2023-11-24T06:15:53.749753+0000","last_became_active":"2023-11-24T06:15:51.170887+0000","last_became_peered":"2023-11-24T06:15:51.170887+0000","last_unstale":"2023-11-24T06:15:53.749753+0000","last_undegraded":"2023-11-24T06:15:53.749753+0000","last_fullsized":"2023-11-24T06:15:53.749753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:24:38.346475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749967+0000","last_change":"2023-11-24T06:15:51.825705+0000","last_active":"2023-11-24T06:15:53.749967+0000","last_peered":"2023-11-24T06:15:53.749967+0000","last_clean":"2023-11-24T06:15:53.749967+0000","last_became_active":"2023-11-24T06:15:51.825372+0000","last_became_peered":"2023-11-24T06:15:51.825372+0000","last_unstale":"2023-11-24T06:15:53.749967+0000","last_undegraded":"2023-11-24T06:15:53.749967+0000","last_fullsized":"2023-11-24T06:15:53.749967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:57:57.197250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.139651+0000","last_change":"2023-11-24T06:15:51.155610+0000","last_active":"2023-11-24T06:15:53.139651+0000","last_peered":"2023-11-24T06:15:53.139651+0000","last_clean":"2023-11-24T06:15:53.139651+0000","last_became_active":"2023-11-24T06:15:51.155318+0000","last_became_peered":"2023-11-24T06:15:51.155318+0000","last_unstale":"2023-11-24T06:15:53.139651+0000","last_undegraded":"2023-11-24T06:15:53.139651+0000","last_fullsized":"2023-11-24T06:15:53.139651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:07.193249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183869+0000","last_change":"2023-11-24T06:15:51.825462+0000","last_active":"2023-11-24T06:15:54.183869+0000","last_peered":"2023-11-24T06:15:54.183869+0000","last_clean":"2023-11-24T06:15:54.183869+0000","last_became_active":"2023-11-24T06:15:51.825148+0000","last_became_peered":"2023-11-24T06:15:51.825148+0000","last_unstale":"2023-11-24T06:15:54.183869+0000","last_undegraded":"2023-11-24T06:15:54.183869+0000","last_fullsized":"2023-11-24T06:15:54.183869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:46.484813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.158814+0000","last_change":"2023-11-24T06:15:51.151498+0000","last_active":"2023-11-24T06:15:52.158814+0000","last_peered":"2023-11-24T06:15:52.158814+0000","last_clean":"2023-11-24T06:15:52.158814+0000","last_became_active":"2023-11-24T06:15:51.151116+0000","last_became_peered":"2023-11-24T06:15:51.151116+0000","last_unstale":"2023-11-24T06:15:52.158814+0000","last_undegraded":"2023-11-24T06:15:52.158814+0000","last_fullsized":"2023-11-24T06:15:52.158814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:32.153844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184490+0000","last_change":"2023-11-24T06:15:51.826667+0000","last_active":"2023-11-24T06:15:54.184490+0000","last_peered":"2023-11-24T06:15:54.184490+0000","last_clean":"2023-11-24T06:15:54.184490+0000","last_became_active":"2023-11-24T06:15:51.826286+0000","last_became_peered":"2023-11-24T06:15:51.826286+0000","last_unstale":"2023-11-24T06:15:54.184490+0000","last_undegraded":"2023-11-24T06:15:54.184490+0000","last_fullsized":"2023-11-24T06:15:54.184490+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:27.816377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.140108+0000","last_change":"2023-11-24T06:15:51.156776+0000","last_active":"2023-11-24T06:15:53.140108+0000","last_peered":"2023-11-24T06:15:53.140108+0000","last_clean":"2023-11-24T06:15:53.140108+0000","last_became_active":"2023-11-24T06:15:51.156313+0000","last_became_peered":"2023-11-24T06:15:51.156313+0000","last_unstale":"2023-11-24T06:15:53.140108+0000","last_undegraded":"2023-11-24T06:15:53.140108+0000","last_fullsized":"2023-11-24T06:15:53.140108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:47:58.118634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185599+0000","last_change":"2023-11-24T06:15:51.825443+0000","last_active":"2023-11-24T06:15:54.185599+0000","last_peered":"2023-11-24T06:15:54.185599+0000","last_clean":"2023-11-24T06:15:54.185599+0000","last_became_active":"2023-11-24T06:15:51.825149+0000","last_became_peered":"2023-11-24T06:15:51.825149+0000","last_unstale":"2023-11-24T06:15:54.185599+0000","last_undegraded":"2023-11-24T06:15:54.185599+0000","last_fullsized":"2023-11-24T06:15:54.185599+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:37.832748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789200+0000","last_change":"2023-11-24T06:15:51.158822+0000","last_active":"2023-11-24T06:15:52.789200+0000","last_peered":"2023-11-24T06:15:52.789200+0000","last_clean":"2023-11-24T06:15:52.789200+0000","last_became_active":"2023-11-24T06:15:51.158544+0000","last_became_peered":"2023-11-24T06:15:51.158544+0000","last_unstale":"2023-11-24T06:15:52.789200+0000","last_undegraded":"2023-11-24T06:15:52.789200+0000","last_fullsized":"2023-11-24T06:15:52.789200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:02.689172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.794289+0000","last_change":"2023-11-24T06:15:51.808175+0000","last_active":"2023-11-24T06:15:52.794289+0000","last_peered":"2023-11-24T06:15:52.794289+0000","last_clean":"2023-11-24T06:15:52.794289+0000","last_became_active":"2023-11-24T06:15:51.807904+0000","last_became_peered":"2023-11-24T06:15:51.807904+0000","last_unstale":"2023-11-24T06:15:52.794289+0000","last_undegraded":"2023-11-24T06:15:52.794289+0000","last_fullsized":"2023-11-24T06:15:52.794289+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:54.525459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.139914+0000","last_change":"2023-11-24T06:15:51.175398+0000","last_active":"2023-11-24T06:15:53.139914+0000","last_peered":"2023-11-24T06:15:53.139914+0000","last_clean":"2023-11-24T06:15:53.139914+0000","last_became_active":"2023-11-24T06:15:51.175039+0000","last_became_peered":"2023-11-24T06:15:51.175039+0000","last_unstale":"2023-11-24T06:15:53.139914+0000","last_undegraded":"2023-11-24T06:15:53.139914+0000","last_fullsized":"2023-11-24T06:15:53.139914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:07.834230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185074+0000","last_change":"2023-11-24T06:15:51.829803+0000","last_active":"2023-11-24T06:15:54.185074+0000","last_peered":"2023-11-24T06:15:54.185074+0000","last_clean":"2023-11-24T06:15:54.185074+0000","last_became_active":"2023-11-24T06:15:51.829417+0000","last_became_peered":"2023-11-24T06:15:51.829417+0000","last_unstale":"2023-11-24T06:15:54.185074+0000","last_undegraded":"2023-11-24T06:15:54.185074+0000","last_fullsized":"2023-11-24T06:15:54.185074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:55:04.397371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:51.810683+0000","last_change":"2023-11-24T06:15:42.021608+0000","last_active":"2023-11-24T06:15:51.810683+0000","last_peered":"2023-11-24T06:15:51.810683+0000","last_clean":"2023-11-24T06:15:51.810683+0000","last_became_active":"2023-11-24T06:15:42.018847+0000","last_became_peered":"2023-11-24T06:15:42.018847+0000","last_unstale":"2023-11-24T06:15:51.810683+0000","last_undegraded":"2023-11-24T06:15:51.810683+0000","last_fullsized":"2023-11-24T06:15:51.810683+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_clean_scrub_stamp":"2023-11-24T06:15:40.694368+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:41.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792437+0000","last_change":"2023-11-24T06:15:51.161936+0000","last_active":"2023-11-24T06:15:52.792437+0000","last_peered":"2023-11-24T06:15:52.792437+0000","last_clean":"2023-11-24T06:15:52.792437+0000","last_became_active":"2023-11-24T06:15:51.161566+0000","last_became_peered":"2023-11-24T06:15:51.161566+0000","last_unstale":"2023-11-24T06:15:52.792437+0000","last_undegraded":"2023-11-24T06:15:52.792437+0000","last_fullsized":"2023-11-24T06:15:52.792437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:23:42.342474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845108+0000","last_change":"2023-11-24T06:15:51.816547+0000","last_active":"2023-11-24T06:15:53.845108+0000","last_peered":"2023-11-24T06:15:53.845108+0000","last_clean":"2023-11-24T06:15:53.845108+0000","last_became_active":"2023-11-24T06:15:51.816231+0000","last_became_peered":"2023-11-24T06:15:51.816231+0000","last_unstale":"2023-11-24T06:15:53.845108+0000","last_undegraded":"2023-11-24T06:15:53.845108+0000","last_fullsized":"2023-11-24T06:15:53.845108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:21:49.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183719+0000","last_change":"2023-11-24T06:15:51.164362+0000","last_active":"2023-11-24T06:15:54.183719+0000","last_peered":"2023-11-24T06:15:54.183719+0000","last_clean":"2023-11-24T06:15:54.183719+0000","last_became_active":"2023-11-24T06:15:51.163606+0000","last_became_peered":"2023-11-24T06:15:51.163606+0000","last_unstale":"2023-11-24T06:15:54.183719+0000","last_undegraded":"2023-11-24T06:15:54.183719+0000","last_fullsized":"2023-11-24T06:15:54.183719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:50:36.901341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.182391+0000","last_change":"2023-11-24T06:15:51.821396+0000","last_active":"2023-11-24T06:15:54.182391+0000","last_peered":"2023-11-24T06:15:54.182391+0000","last_clean":"2023-11-24T06:15:54.182391+0000","last_became_active":"2023-11-24T06:15:51.821096+0000","last_became_peered":"2023-11-24T06:15:51.821096+0000","last_unstale":"2023-11-24T06:15:54.182391+0000","last_undegraded":"2023-11-24T06:15:54.182391+0000","last_fullsized":"2023-11-24T06:15:54.182391+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:17.158925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.140207+0000","last_change":"2023-11-24T06:15:51.156378+0000","last_active":"2023-11-24T06:15:53.140207+0000","last_peered":"2023-11-24T06:15:53.140207+0000","last_clean":"2023-11-24T06:15:53.140207+0000","last_became_active":"2023-11-24T06:15:51.156003+0000","last_became_peered":"2023-11-24T06:15:51.156003+0000","last_unstale":"2023-11-24T06:15:53.140207+0000","last_undegraded":"2023-11-24T06:15:53.140207+0000","last_fullsized":"2023-11-24T06:15:53.140207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:31.267818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788625+0000","last_change":"2023-11-24T06:15:51.810053+0000","last_active":"2023-11-24T06:15:52.788625+0000","last_peered":"2023-11-24T06:15:52.788625+0000","last_clean":"2023-11-24T06:15:52.788625+0000","last_became_active":"2023-11-24T06:15:51.806554+0000","last_became_peered":"2023-11-24T06:15:51.806554+0000","last_unstale":"2023-11-24T06:15:52.788625+0000","last_undegraded":"2023-11-24T06:15:52.788625+0000","last_fullsized":"2023-11-24T06:15:52.788625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:01.976941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793809+0000","last_change":"2023-11-24T06:15:51.170982+0000","last_active":"2023-11-24T06:15:52.793809+0000","last_peered":"2023-11-24T06:15:52.793809+0000","last_clean":"2023-11-24T06:15:52.793809+0000","last_became_active":"2023-11-24T06:15:51.170654+0000","last_became_peered":"2023-11-24T06:15:51.170654+0000","last_unstale":"2023-11-24T06:15:52.793809+0000","last_undegraded":"2023-11-24T06:15:52.793809+0000","last_fullsized":"2023-11-24T06:15:52.793809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:48.317425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.792676+0000","last_change":"2023-11-24T06:15:51.823915+0000","last_active":"2023-11-24T06:15:52.792676+0000","last_peered":"2023-11-24T06:15:52.792676+0000","last_clean":"2023-11-24T06:15:52.792676+0000","last_became_active":"2023-11-24T06:15:51.823654+0000","last_became_peered":"2023-11-24T06:15:51.823654+0000","last_unstale":"2023-11-24T06:15:52.792676+0000","last_undegraded":"2023-11-24T06:15:52.792676+0000","last_fullsized":"2023-11-24T06:15:52.792676+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:17:01.717621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.789034+0000","last_change":"2023-11-24T06:15:51.154171+0000","last_active":"2023-11-24T06:15:52.789034+0000","last_peered":"2023-11-24T06:15:52.789034+0000","last_clean":"2023-11-24T06:15:52.789034+0000","last_became_active":"2023-11-24T06:15:51.153885+0000","last_became_peered":"2023-11-24T06:15:51.153885+0000","last_unstale":"2023-11-24T06:15:52.789034+0000","last_undegraded":"2023-11-24T06:15:52.789034+0000","last_fullsized":"2023-11-24T06:15:52.789034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:36:33.590350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793463+0000","last_change":"2023-11-24T06:15:51.809270+0000","last_active":"2023-11-24T06:15:52.793463+0000","last_peered":"2023-11-24T06:15:52.793463+0000","last_clean":"2023-11-24T06:15:52.793463+0000","last_became_active":"2023-11-24T06:15:51.808994+0000","last_became_peered":"2023-11-24T06:15:51.808994+0000","last_unstale":"2023-11-24T06:15:52.793463+0000","last_undegraded":"2023-11-24T06:15:52.793463+0000","last_fullsized":"2023-11-24T06:15:52.793463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:00:54.543519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185536+0000","last_change":"2023-11-24T06:15:51.166677+0000","last_active":"2023-11-24T06:15:54.185536+0000","last_peered":"2023-11-24T06:15:54.185536+0000","last_clean":"2023-11-24T06:15:54.185536+0000","last_became_active":"2023-11-24T06:15:51.164374+0000","last_became_peered":"2023-11-24T06:15:51.164374+0000","last_unstale":"2023-11-24T06:15:54.185536+0000","last_undegraded":"2023-11-24T06:15:54.185536+0000","last_fullsized":"2023-11-24T06:15:54.185536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:00:50.542922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788766+0000","last_change":"2023-11-24T06:15:51.808320+0000","last_active":"2023-11-24T06:15:52.788766+0000","last_peered":"2023-11-24T06:15:52.788766+0000","last_clean":"2023-11-24T06:15:52.788766+0000","last_became_active":"2023-11-24T06:15:51.804832+0000","last_became_peered":"2023-11-24T06:15:51.804832+0000","last_unstale":"2023-11-24T06:15:52.788766+0000","last_undegraded":"2023-11-24T06:15:52.788766+0000","last_fullsized":"2023-11-24T06:15:52.788766+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:21:49.599954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.119685+0000","last_change":"2023-11-24T06:15:51.168020+0000","last_active":"2023-11-24T06:15:54.119685+0000","last_peered":"2023-11-24T06:15:54.119685+0000","last_clean":"2023-11-24T06:15:54.119685+0000","last_became_active":"2023-11-24T06:15:51.167664+0000","last_became_peered":"2023-11-24T06:15:51.167664+0000","last_unstale":"2023-11-24T06:15:54.119685+0000","last_undegraded":"2023-11-24T06:15:54.119685+0000","last_fullsized":"2023-11-24T06:15:54.119685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:17:00.705564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845799+0000","last_change":"2023-11-24T06:15:51.816931+0000","last_active":"2023-11-24T06:15:53.845799+0000","last_peered":"2023-11-24T06:15:53.845799+0000","last_clean":"2023-11-24T06:15:53.845799+0000","last_became_active":"2023-11-24T06:15:51.816611+0000","last_became_peered":"2023-11-24T06:15:51.816611+0000","last_unstale":"2023-11-24T06:15:53.845799+0000","last_undegraded":"2023-11-24T06:15:53.845799+0000","last_fullsized":"2023-11-24T06:15:53.845799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:05:27.816377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.750733+0000","last_change":"2023-11-24T06:15:51.172657+0000","last_active":"2023-11-24T06:15:53.750733+0000","last_peered":"2023-11-24T06:15:53.750733+0000","last_clean":"2023-11-24T06:15:53.750733+0000","last_became_active":"2023-11-24T06:15:51.172152+0000","last_became_peered":"2023-11-24T06:15:51.172152+0000","last_unstale":"2023-11-24T06:15:53.750733+0000","last_undegraded":"2023-11-24T06:15:53.750733+0000","last_fullsized":"2023-11-24T06:15:53.750733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:23:43.714712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183046+0000","last_change":"2023-11-24T06:15:51.815689+0000","last_active":"2023-11-24T06:15:54.183046+0000","last_peered":"2023-11-24T06:15:54.183046+0000","last_clean":"2023-11-24T06:15:54.183046+0000","last_became_active":"2023-11-24T06:15:51.814742+0000","last_became_peered":"2023-11-24T06:15:51.814742+0000","last_unstale":"2023-11-24T06:15:54.183046+0000","last_undegraded":"2023-11-24T06:15:54.183046+0000","last_fullsized":"2023-11-24T06:15:54.183046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:05.782767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.183694+0000","last_change":"2023-11-24T06:15:51.164574+0000","last_active":"2023-11-24T06:15:54.183694+0000","last_peered":"2023-11-24T06:15:54.183694+0000","last_clean":"2023-11-24T06:15:54.183694+0000","last_became_active":"2023-11-24T06:15:51.163261+0000","last_became_peered":"2023-11-24T06:15:51.163261+0000","last_unstale":"2023-11-24T06:15:54.183694+0000","last_undegraded":"2023-11-24T06:15:54.183694+0000","last_fullsized":"2023-11-24T06:15:54.183694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:19:30.567064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.791508+0000","last_change":"2023-11-24T06:15:51.808104+0000","last_active":"2023-11-24T06:15:52.791508+0000","last_peered":"2023-11-24T06:15:52.791508+0000","last_clean":"2023-11-24T06:15:52.791508+0000","last_became_active":"2023-11-24T06:15:51.807649+0000","last_became_peered":"2023-11-24T06:15:51.807649+0000","last_unstale":"2023-11-24T06:15:52.791508+0000","last_undegraded":"2023-11-24T06:15:52.791508+0000","last_fullsized":"2023-11-24T06:15:52.791508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:41:47.469041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788424+0000","last_change":"2023-11-24T06:15:51.160561+0000","last_active":"2023-11-24T06:15:52.788424+0000","last_peered":"2023-11-24T06:15:52.788424+0000","last_clean":"2023-11-24T06:15:52.788424+0000","last_became_active":"2023-11-24T06:15:51.160266+0000","last_became_peered":"2023-11-24T06:15:51.160266+0000","last_unstale":"2023-11-24T06:15:52.788424+0000","last_undegraded":"2023-11-24T06:15:52.788424+0000","last_fullsized":"2023-11-24T06:15:52.788424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:09:38.300549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845022+0000","last_change":"2023-11-24T06:15:51.806779+0000","last_active":"2023-11-24T06:15:53.845022+0000","last_peered":"2023-11-24T06:15:53.845022+0000","last_clean":"2023-11-24T06:15:53.845022+0000","last_became_active":"2023-11-24T06:15:51.806429+0000","last_became_peered":"2023-11-24T06:15:51.806429+0000","last_unstale":"2023-11-24T06:15:53.845022+0000","last_undegraded":"2023-11-24T06:15:53.845022+0000","last_fullsized":"2023-11-24T06:15:53.845022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:11:01.976941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.788974+0000","last_change":"2023-11-24T06:15:51.157354+0000","last_active":"2023-11-24T06:15:52.788974+0000","last_peered":"2023-11-24T06:15:52.788974+0000","last_clean":"2023-11-24T06:15:52.788974+0000","last_became_active":"2023-11-24T06:15:51.156981+0000","last_became_peered":"2023-11-24T06:15:51.156981+0000","last_unstale":"2023-11-24T06:15:52.788974+0000","last_undegraded":"2023-11-24T06:15:52.788974+0000","last_fullsized":"2023-11-24T06:15:52.788974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:19:30.567064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845730+0000","last_change":"2023-11-24T06:15:51.823678+0000","last_active":"2023-11-24T06:15:53.845730+0000","last_peered":"2023-11-24T06:15:53.845730+0000","last_clean":"2023-11-24T06:15:53.845730+0000","last_became_active":"2023-11-24T06:15:51.823372+0000","last_became_peered":"2023-11-24T06:15:51.823372+0000","last_unstale":"2023-11-24T06:15:53.845730+0000","last_undegraded":"2023-11-24T06:15:53.845730+0000","last_fullsized":"2023-11-24T06:15:53.845730+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:46.484813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184891+0000","last_change":"2023-11-24T06:15:51.167493+0000","last_active":"2023-11-24T06:15:54.184891+0000","last_peered":"2023-11-24T06:15:54.184891+0000","last_clean":"2023-11-24T06:15:54.184891+0000","last_became_active":"2023-11-24T06:15:51.167110+0000","last_became_peered":"2023-11-24T06:15:51.167110+0000","last_unstale":"2023-11-24T06:15:54.184891+0000","last_undegraded":"2023-11-24T06:15:54.184891+0000","last_fullsized":"2023-11-24T06:15:54.184891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:59.444337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.749958+0000","last_change":"2023-11-24T06:15:51.816423+0000","last_active":"2023-11-24T06:15:53.749958+0000","last_peered":"2023-11-24T06:15:53.749958+0000","last_clean":"2023-11-24T06:15:53.749958+0000","last_became_active":"2023-11-24T06:15:51.816122+0000","last_became_peered":"2023-11-24T06:15:51.816122+0000","last_unstale":"2023-11-24T06:15:53.749958+0000","last_undegraded":"2023-11-24T06:15:53.749958+0000","last_fullsized":"2023-11-24T06:15:53.749958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:50.460127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793429+0000","last_change":"2023-11-24T06:15:51.803878+0000","last_active":"2023-11-24T06:15:52.793429+0000","last_peered":"2023-11-24T06:15:52.793429+0000","last_clean":"2023-11-24T06:15:52.793429+0000","last_became_active":"2023-11-24T06:15:51.803472+0000","last_became_peered":"2023-11-24T06:15:51.803472+0000","last_unstale":"2023-11-24T06:15:52.793429+0000","last_undegraded":"2023-11-24T06:15:52.793429+0000","last_fullsized":"2023-11-24T06:15:52.793429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:23.431671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.184531+0000","last_change":"2023-11-24T06:15:51.177433+0000","last_active":"2023-11-24T06:15:54.184531+0000","last_peered":"2023-11-24T06:15:54.184531+0000","last_clean":"2023-11-24T06:15:54.184531+0000","last_became_active":"2023-11-24T06:15:51.176987+0000","last_became_peered":"2023-11-24T06:15:51.176987+0000","last_unstale":"2023-11-24T06:15:54.184531+0000","last_undegraded":"2023-11-24T06:15:54.184531+0000","last_fullsized":"2023-11-24T06:15:54.184531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:18.981340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:52.793828+0000","last_change":"2023-11-24T06:15:51.815123+0000","last_active":"2023-11-24T06:15:52.793828+0000","last_peered":"2023-11-24T06:15:52.793828+0000","last_clean":"2023-11-24T06:15:52.793828+0000","last_became_active":"2023-11-24T06:15:51.814778+0000","last_became_peered":"2023-11-24T06:15:51.814778+0000","last_unstale":"2023-11-24T06:15:52.793828+0000","last_undegraded":"2023-11-24T06:15:52.793828+0000","last_fullsized":"2023-11-24T06:15:52.793828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:12.167390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.185069+0000","last_change":"2023-11-24T06:15:51.169354+0000","last_active":"2023-11-24T06:15:54.185069+0000","last_peered":"2023-11-24T06:15:54.185069+0000","last_clean":"2023-11-24T06:15:54.185069+0000","last_became_active":"2023-11-24T06:15:51.168916+0000","last_became_peered":"2023-11-24T06:15:51.168916+0000","last_unstale":"2023-11-24T06:15:54.185069+0000","last_undegraded":"2023-11-24T06:15:54.185069+0000","last_fullsized":"2023-11-24T06:15:54.185069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:15.261716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.182984+0000","last_change":"2023-11-24T06:15:51.818739+0000","last_active":"2023-11-24T06:15:54.182984+0000","last_peered":"2023-11-24T06:15:54.182984+0000","last_clean":"2023-11-24T06:15:54.182984+0000","last_became_active":"2023-11-24T06:15:51.817316+0000","last_became_peered":"2023-11-24T06:15:51.817316+0000","last_unstale":"2023-11-24T06:15:54.182984+0000","last_undegraded":"2023-11-24T06:15:54.182984+0000","last_fullsized":"2023-11-24T06:15:54.182984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:12:12.520019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845707+0000","last_change":"2023-11-24T06:15:51.152517+0000","last_active":"2023-11-24T06:15:53.845707+0000","last_peered":"2023-11-24T06:15:53.845707+0000","last_clean":"2023-11-24T06:15:53.845707+0000","last_became_active":"2023-11-24T06:15:51.152218+0000","last_became_peered":"2023-11-24T06:15:51.152218+0000","last_unstale":"2023-11-24T06:15:53.845707+0000","last_undegraded":"2023-11-24T06:15:53.845707+0000","last_fullsized":"2023-11-24T06:15:53.845707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:59.444337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T06:15:53.845977+0000","last_change":"2023-11-24T06:15:51.817856+0000","last_active":"2023-11-24T06:15:53.845977+0000","last_peered":"2023-11-24T06:15:53.845977+0000","last_clean":"2023-11-24T06:15:53.845977+0000","last_became_active":"2023-11-24T06:15:51.817558+0000","last_became_peered":"2023-11-24T06:15:51.817558+0000","last_unstale":"2023-11-24T06:15:53.845977+0000","last_undegraded":"2023-11-24T06:15:53.845977+0000","last_fullsized":"2023-11-24T06:15:53.845977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:51.110735+0000","last_clean_scrub_stamp":"2023-11-24T06:15:51.110735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:22:47.291087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T06:15:54.129254+0000","last_change":"2023-11-24T06:15:51.157608+0000","last_active":"2023-11-24T06:15:54.129254+0000","last_peered":"2023-11-24T06:15:54.129254+0000","last_clean":"2023-11-24T06:15:54.129254+0000","last_became_active":"2023-11-24T06:15:51.157330+0000","last_became_peered":"2023-11-24T06:15:51.157330+0000","last_unstale":"2023-11-24T06:15:54.129254+0000","last_undegraded":"2023-11-24T06:15:54.129254+0000","last_fullsized":"2023-11-24T06:15:54.129254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:50.098678+0000","last_clean_scrub_stamp":"2023-11-24T06:15:50.098678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:39:05.215342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4339999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4690000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.637}]}]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":78220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6060000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8220000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":79706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7410000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4890000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.673}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.161}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7290000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4820000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6880000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":78184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2829999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.032}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3680000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.573}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":233472,"data_stored":79706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542151,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":221184,"data_stored":78094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":78254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":668381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.165}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:16:02.732 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T06:16:02.732 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-24T06:16:02.732 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-24T06:16:02.733 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-24T06:16:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.988+0000 7f98d15c1640 1 Processor -- start 2023-11-24T06:16:02.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.989+0000 7f98d15c1640 1 -- start start 2023-11-24T06:16:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.990+0000 7f98d15c1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98cc0cf500 0x7f98cc0cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.990+0000 7f98d15c1640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc140900 con 0x7f98cc13dfd0 2023-11-24T06:16:02.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.990+0000 7f98d15c1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98cc15fa70 con 0x7f98cc0cf050 2023-11-24T06:16:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.990+0000 7f98d15c1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc15c580 con 0x7f98cc0cf500 2023-11-24T06:16:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.991+0000 7f98cbfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f98cc13dfd0 0x7f98cc15be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37066/0 (socket says 172.21.15.5:37066) 2023-11-24T06:16:02.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.991+0000 7f98cbfff640 1 -- 172.21.15.5:0/734499856 learned_addr learned my addr 172.21.15.5:0/734499856 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.991+0000 7f98d0dc0640 1 --2- 172.21.15.5:0/734499856 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98cc0cf500 0x7f98cc0cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.991+0000 7f98cb7fe640 1 -- 172.21.15.5:0/734499856 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585849209 0 0) 0x7f98cc140900 con 0x7f98cc13dfd0 2023-11-24T06:16:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.991+0000 7f98cb7fe640 1 -- 172.21.15.5:0/734499856 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98b8003520 con 0x7f98cc13dfd0 2023-11-24T06:16:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.991+0000 7f98cb7fe640 1 -- 172.21.15.5:0/734499856 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 967585911 0 0) 0x7f98b8003520 con 0x7f98cc13dfd0 2023-11-24T06:16:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.991+0000 7f98cb7fe640 1 -- 172.21.15.5:0/734499856 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98cc0cf500 msgr2=0x7f98cc0cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.991+0000 7f98cb7fe640 1 --2- 172.21.15.5:0/734499856 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98cc0cf500 0x7f98cc0cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.991+0000 7f98cb7fe640 1 -- 172.21.15.5:0/734499856 >> v1:172.21.15.6:6789/0 conn(0x7f98cc0cf050 legacy=0x7f98cc0cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.991+0000 7f98cb7fe640 1 -- 172.21.15.5:0/734499856 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc0cf950 con 0x7f98cc13dfd0 2023-11-24T06:16:02.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.991+0000 7f98cb7fe640 1 -- 172.21.15.5:0/734499856 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f98bc001690 con 0x7f98cc13dfd0 2023-11-24T06:16:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.991+0000 7f98cb7fe640 1 -- 172.21.15.5:0/734499856 >> v1:172.21.15.5:6789/0 conn(0x7f98cc13dfd0 legacy=0x7f98cc15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.992+0000 7f98cb7fe640 1 --2- 172.21.15.5:0/734499856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98b8003a10 0x7f98b8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.992+0000 7f98cb7fe640 1 --2- 172.21.15.5:0/734499856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98b80047c0 0x7f98b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.992+0000 7f98cb7fe640 1 --2- 172.21.15.5:0/734499856 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98cc0cf500 0x7f98b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.992+0000 7f98cb7fe640 1 -- 172.21.15.5:0/734499856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc15c580 con 0x7f98b80047c0 2023-11-24T06:16:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.992+0000 7f98cb7fe640 1 -- 172.21.15.5:0/734499856 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b800ba80 con 0x7f98b8003a10 2023-11-24T06:16:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.992+0000 7f98cb7fe640 1 -- 172.21.15.5:0/734499856 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b800bc40 con 0x7f98cc0cf500 2023-11-24T06:16:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.992+0000 7f98c3fff640 1 --2- 172.21.15.5:0/734499856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98b80047c0 0x7f98b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.992+0000 7f98cbfff640 1 --2- 172.21.15.5:0/734499856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98b8003a10 0x7f98b8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.992+0000 7f98c3fff640 1 -- 172.21.15.5:0/734499856 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98cc0cf500 msgr2=0x7f98b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.992+0000 7f98c3fff640 1 --2- 172.21.15.5:0/734499856 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98cc0cf500 0x7f98b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.992+0000 7f98c3fff640 1 -- 172.21.15.5:0/734499856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98b8003a10 msgr2=0x7f98b8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.992+0000 7f98c3fff640 1 --2- 172.21.15.5:0/734499856 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98b8003a10 0x7f98b8003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.992+0000 7f98c3fff640 1 -- 172.21.15.5:0/734499856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b8003520 con 0x7f98b80047c0 2023-11-24T06:16:02.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.993+0000 7f98c3fff640 1 --2- 172.21.15.5:0/734499856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98b80047c0 0x7f98b8004c40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f98b4001750 tx=0x7f98b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361f6a05b856728d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:02.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.994+0000 7f98cb7fe640 1 -- 172.21.15.5:0/734499856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b40052d0 con 0x7f98b80047c0 2023-11-24T06:16:02.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.994+0000 7f98cb7fe640 1 -- 172.21.15.5:0/734499856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b402e050 con 0x7f98b80047c0 2023-11-24T06:16:02.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.994+0000 7f98cb7fe640 1 -- 172.21.15.5:0/734499856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b40035f0 con 0x7f98b80047c0 2023-11-24T06:16:02.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.994+0000 7f98d15c1640 1 -- 172.21.15.5:0/734499856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98b80047c0 msgr2=0x7f98b8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:02.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.994+0000 7f98d15c1640 1 --2- 172.21.15.5:0/734499856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98b80047c0 0x7f98b8004c40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f98b4001750 tx=0x7f98b80057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:02.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.994+0000 7f98d0dc0640 1 -- 172.21.15.5:0/734499856 reap_dead start 2023-11-24T06:16:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.994+0000 7f98d15c1640 1 -- 172.21.15.5:0/734499856 shutdown_connections 2023-11-24T06:16:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.994+0000 7f98d15c1640 1 -- 172.21.15.5:0/734499856 >> 172.21.15.5:0/734499856 conn(0x7f98cc06a860 msgr2=0x7f98cc06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.994+0000 7f98d15c1640 1 -- 172.21.15.5:0/734499856 shutdown_connections 2023-11-24T06:16:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.994+0000 7f98d15c1640 1 -- 172.21.15.5:0/734499856 wait complete. 2023-11-24T06:16:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.996+0000 7f98d15c1640 1 Processor -- start 2023-11-24T06:16:03.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.996+0000 7f98d15c1640 1 -- start start 2023-11-24T06:16:03.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.997+0000 7f98d15c1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98b80047c0 0x7f98cc155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.997+0000 7f98d15c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc13e440 0x7f98cc1391b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.997+0000 7f98d15c1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98cc15ae50 0x7f98cc13a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.997+0000 7f98d15c1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc1396f0 con 0x7f98cc13e440 2023-11-24T06:16:03.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.997+0000 7f98d15c1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc13ea00 con 0x7f98b80047c0 2023-11-24T06:16:03.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.997+0000 7f98d15c1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc13ece0 con 0x7f98cc15ae50 2023-11-24T06:16:03.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.997+0000 7f98c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc13e440 0x7f98cc1391b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.998+0000 7f98c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc13e440 0x7f98cc1391b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41478/0 (socket says 172.21.15.5:41478) 2023-11-24T06:16:03.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.998+0000 7f98c3fff640 1 -- 172.21.15.5:0/3317084015 learned_addr learned my addr 172.21.15.5:0/3317084015 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:03.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.998+0000 7f98cbfff640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98b80047c0 0x7f98cc155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.998+0000 7f98d0dc0640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98cc15ae50 0x7f98cc13a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.998+0000 7f98c3fff640 1 -- 172.21.15.5:0/3317084015 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98cc15ae50 msgr2=0x7f98cc13a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:03.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.998+0000 7f98c3fff640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98cc15ae50 0x7f98cc13a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.998+0000 7f98c3fff640 1 -- 172.21.15.5:0/3317084015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98b80047c0 msgr2=0x7f98cc155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:03.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.998+0000 7f98c3fff640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98b80047c0 0x7f98cc155c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.998+0000 7f98c3fff640 1 -- 172.21.15.5:0/3317084015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc13efc0 con 0x7f98cc13e440 2023-11-24T06:16:03.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.998+0000 7f98d0dc0640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98cc15ae50 0x7f98cc13a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:03.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.998+0000 7f98c3fff640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc13e440 0x7f98cc1391b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f98b4008fd0 tx=0x7f98b4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:03.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.999+0000 7f98c97fa640 1 -- 172.21.15.5:0/3317084015 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b40076c0 con 0x7f98cc13e440 2023-11-24T06:16:03.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.999+0000 7f98c97fa640 1 -- 172.21.15.5:0/3317084015 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b4007a30 con 0x7f98cc13e440 2023-11-24T06:16:03.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.999+0000 7f98c97fa640 1 -- 172.21.15.5:0/3317084015 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b4006c00 con 0x7f98cc13e440 2023-11-24T06:16:03.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.999+0000 7f98d15c1640 1 -- 172.21.15.5:0/3317084015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc13f2a0 con 0x7f98cc13e440 2023-11-24T06:16:03.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:02.999+0000 7f98d15c1640 1 -- 172.21.15.5:0/3317084015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc15b300 con 0x7f98cc13e440 2023-11-24T06:16:03.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.000+0000 7f98d15c1640 1 -- 172.21.15.5:0/3317084015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc15b490 con 0x7f98cc13e440 2023-11-24T06:16:03.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.001+0000 7f98c97fa640 1 -- 172.21.15.5:0/3317084015 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98b40035f0 con 0x7f98cc13e440 2023-11-24T06:16:03.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.001+0000 7f98c97fa640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f98ac06ada0 0x7f98ac06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.002+0000 7f98c97fa640 1 -- 172.21.15.5:0/3317084015 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98b40ab7d0 con 0x7f98cc13e440 2023-11-24T06:16:03.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.002+0000 7f98cbfff640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f98ac06ada0 0x7f98ac06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.002+0000 7f98cbfff640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f98ac06ada0 0x7f98ac06d260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f98cc06dec0 tx=0x7f98bc000f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:03.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.006+0000 7f98c97fa640 1 -- 172.21.15.5:0/3317084015 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98b407a310 con 0x7f98cc13e440 2023-11-24T06:16:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.231+0000 7f98d15c1640 1 -- 172.21.15.5:0/3317084015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f98cc04e840 con 0x7f98cc13e440 2023-11-24T06:16:03.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.231+0000 7f98c97fa640 1 -- 172.21.15.5:0/3317084015 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f98b4079ea0 con 0x7f98cc13e440 2023-11-24T06:16:03.234 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:03.235 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-24T06:16:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.234+0000 7f98d15c1640 1 -- 172.21.15.5:0/3317084015 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f98ac06ada0 msgr2=0x7f98ac06d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.234+0000 7f98d15c1640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f98ac06ada0 0x7f98ac06d260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f98cc06dec0 tx=0x7f98bc000f90 comp rx=0 tx=0).stop 2023-11-24T06:16:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.234+0000 7f98d15c1640 1 -- 172.21.15.5:0/3317084015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc13e440 msgr2=0x7f98cc1391b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.234+0000 7f98d15c1640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc13e440 0x7f98cc1391b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f98b4008fd0 tx=0x7f98b4001120 comp rx=0 tx=0).stop 2023-11-24T06:16:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.234+0000 7f98d15c1640 1 -- 172.21.15.5:0/3317084015 shutdown_connections 2023-11-24T06:16:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.234+0000 7f98d15c1640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f98ac06ada0 0x7f98ac06d260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.234+0000 7f98d15c1640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f98cc15ae50 0x7f98cc13a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.234+0000 7f98d15c1640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc13e440 0x7f98cc1391b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.234+0000 7f98d15c1640 1 --2- 172.21.15.5:0/3317084015 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f98b80047c0 0x7f98cc155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.234+0000 7f98d15c1640 1 -- 172.21.15.5:0/3317084015 >> 172.21.15.5:0/3317084015 conn(0x7f98cc06a860 msgr2=0x7f98cc0afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:03.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.234+0000 7f98d15c1640 1 -- 172.21.15.5:0/3317084015 shutdown_connections 2023-11-24T06:16:03.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.234+0000 7f98d15c1640 1 -- 172.21.15.5:0/3317084015 wait complete. 2023-11-24T06:16:03.254 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-24T06:16:03.255 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:03.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.469+0000 7f444ffff640 1 Processor -- start 2023-11-24T06:16:03.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.469+0000 7f444ffff640 1 -- start start 2023-11-24T06:16:03.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.471+0000 7f444ffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f445013dfd0 0x7f445015be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.471+0000 7f444ffff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f44500cf050 0x7f44500cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.471+0000 7f444ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f445006c130 0x7f445006c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.471+0000 7f444ffff640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44500cfa10 con 0x7f445006c130 2023-11-24T06:16:03.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.472+0000 7f444ffff640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445015fa70 con 0x7f44500cf050 2023-11-24T06:16:03.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.472+0000 7f444ffff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445015ee60 con 0x7f445013dfd0 2023-11-24T06:16:03.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.472+0000 7f444f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f445006c130 0x7f445006c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.472+0000 7f444f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f445006c130 0x7f445006c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41492/0 (socket says 172.21.15.5:41492) 2023-11-24T06:16:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.472+0000 7f444f7fe640 1 -- 172.21.15.5:0/2379066569 learned_addr learned my addr 172.21.15.5:0/2379066569 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.472+0000 7f4447fff640 1 --2- 172.21.15.5:0/2379066569 >> v2:172.21.15.6:3300/0 conn(0x7f44500cf050 0x7f44500cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.472+0000 7f444effd640 1 --2- 172.21.15.5:0/2379066569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f445013dfd0 0x7f445015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.472+0000 7f444f7fe640 1 -- 172.21.15.5:0/2379066569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f445013dfd0 msgr2=0x7f445015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:03.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.472+0000 7f444f7fe640 1 --2- 172.21.15.5:0/2379066569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f445013dfd0 0x7f445015be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.472+0000 7f444f7fe640 1 -- 172.21.15.5:0/2379066569 >> v2:172.21.15.6:3300/0 conn(0x7f44500cf050 msgr2=0x7f44500cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:03.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.472+0000 7f444f7fe640 1 --2- 172.21.15.5:0/2379066569 >> v2:172.21.15.6:3300/0 conn(0x7f44500cf050 0x7f44500cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.472+0000 7f444f7fe640 1 -- 172.21.15.5:0/2379066569 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445015f140 con 0x7f445006c130 2023-11-24T06:16:03.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.472+0000 7f4447fff640 1 --2- 172.21.15.5:0/2379066569 >> v2:172.21.15.6:3300/0 conn(0x7f44500cf050 0x7f44500cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:16:03.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.474+0000 7f444f7fe640 1 --2- 172.21.15.5:0/2379066569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445006c130 0x7f445006c580 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f44500cfc40 tx=0x7f444802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d7fa41637f34f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:03.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.474+0000 7f444e7fc640 1 -- 172.21.15.5:0/2379066569 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444802ca80 con 0x7f445006c130 2023-11-24T06:16:03.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.474+0000 7f444e7fc640 1 -- 172.21.15.5:0/2379066569 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444802cbe0 con 0x7f445006c130 2023-11-24T06:16:03.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.474+0000 7f444e7fc640 1 -- 172.21.15.5:0/2379066569 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44480343f0 con 0x7f445006c130 2023-11-24T06:16:03.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.475+0000 7f444ffff640 1 -- 172.21.15.5:0/2379066569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445006c130 msgr2=0x7f445006c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:03.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.475+0000 7f444ffff640 1 --2- 172.21.15.5:0/2379066569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445006c130 0x7f445006c580 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f44500cfc40 tx=0x7f444802bee0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.475+0000 7f444ffff640 1 -- 172.21.15.5:0/2379066569 shutdown_connections 2023-11-24T06:16:03.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.475+0000 7f444ffff640 1 --2- 172.21.15.5:0/2379066569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445006c130 0x7f445006c580 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.475+0000 7f444ffff640 1 --2- 172.21.15.5:0/2379066569 >> v2:172.21.15.6:3300/0 conn(0x7f44500cf050 0x7f44500cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.475+0000 7f444ffff640 1 --2- 172.21.15.5:0/2379066569 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f445013dfd0 0x7f445015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.475+0000 7f444ffff640 1 -- 172.21.15.5:0/2379066569 >> 172.21.15.5:0/2379066569 conn(0x7f445006a860 msgr2=0x7f445006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:03.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.475+0000 7f444ffff640 1 -- 172.21.15.5:0/2379066569 shutdown_connections 2023-11-24T06:16:03.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.475+0000 7f444ffff640 1 -- 172.21.15.5:0/2379066569 wait complete. 2023-11-24T06:16:03.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.477+0000 7f444ffff640 1 Processor -- start 2023-11-24T06:16:03.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.477+0000 7f444ffff640 1 -- start start 2023-11-24T06:16:03.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.478+0000 7f444ffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f445006c130 0x7f44501f2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.478+0000 7f444ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44500cf050 0x7f4450156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.478+0000 7f444ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445013dfd0 0x7f445015cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.478+0000 7f444ffff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445013f840 con 0x7f445013dfd0 2023-11-24T06:16:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.478+0000 7f444ffff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4450134c10 con 0x7f44500cf050 2023-11-24T06:16:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.478+0000 7f444ffff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4450156dc0 con 0x7f445006c130 2023-11-24T06:16:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.478+0000 7f444f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445013dfd0 0x7f445015cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.478+0000 7f444f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445013dfd0 0x7f445015cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41504/0 (socket says 172.21.15.5:41504) 2023-11-24T06:16:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.478+0000 7f444f7fe640 1 -- 172.21.15.5:0/1557652052 learned_addr learned my addr 172.21.15.5:0/1557652052 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.479+0000 7f4447fff640 1 --2- 172.21.15.5:0/1557652052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44500cf050 0x7f4450156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.479+0000 7f444effd640 1 --2- 172.21.15.5:0/1557652052 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f445006c130 0x7f44501f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.479+0000 7f444f7fe640 1 -- 172.21.15.5:0/1557652052 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f445006c130 msgr2=0x7f44501f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:03.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.479+0000 7f444f7fe640 1 --2- 172.21.15.5:0/1557652052 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f445006c130 0x7f44501f2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.479+0000 7f444f7fe640 1 -- 172.21.15.5:0/1557652052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44500cf050 msgr2=0x7f4450156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:03.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.479+0000 7f444f7fe640 1 --2- 172.21.15.5:0/1557652052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44500cf050 0x7f4450156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.479+0000 7f444f7fe640 1 -- 172.21.15.5:0/1557652052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4448009b30 con 0x7f445013dfd0 2023-11-24T06:16:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.479+0000 7f444f7fe640 1 --2- 172.21.15.5:0/1557652052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445013dfd0 0x7f445015cc90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4448009f40 tx=0x7f4448004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.479+0000 7f44477fe640 1 -- 172.21.15.5:0/1557652052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448009510 con 0x7f445013dfd0 2023-11-24T06:16:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.480+0000 7f44477fe640 1 -- 172.21.15.5:0/1557652052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444803e450 con 0x7f445013dfd0 2023-11-24T06:16:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.480+0000 7f44477fe640 1 -- 172.21.15.5:0/1557652052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444803e800 con 0x7f445013dfd0 2023-11-24T06:16:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.480+0000 7f444ffff640 1 -- 172.21.15.5:0/1557652052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445013f560 con 0x7f445013dfd0 2023-11-24T06:16:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.480+0000 7f444ffff640 1 -- 172.21.15.5:0/1557652052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4450156080 con 0x7f445013dfd0 2023-11-24T06:16:03.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.481+0000 7f444ffff640 1 -- 172.21.15.5:0/1557652052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4450155ba0 con 0x7f445013dfd0 2023-11-24T06:16:03.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.482+0000 7f44477fe640 1 -- 172.21.15.5:0/1557652052 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f444802cd50 con 0x7f445013dfd0 2023-11-24T06:16:03.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.483+0000 7f44477fe640 1 --2- 172.21.15.5:0/1557652052 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f443006ae40 0x7f443006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.483+0000 7f44477fe640 1 -- 172.21.15.5:0/1557652052 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44480ac140 con 0x7f445013dfd0 2023-11-24T06:16:03.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.483+0000 7f444effd640 1 --2- 172.21.15.5:0/1557652052 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f443006ae40 0x7f443006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.484+0000 7f444effd640 1 --2- 172.21.15.5:0/1557652052 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f443006ae40 0x7f443006d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f44400099b0 tx=0x7f4440008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:03.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.487+0000 7f44477fe640 1 -- 172.21.15.5:0/1557652052 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4448077150 con 0x7f445013dfd0 2023-11-24T06:16:03.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.673+0000 7f444ffff640 1 -- 172.21.15.5:0/1557652052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f445004e840 con 0x7f445013dfd0 2023-11-24T06:16:03.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.676+0000 7f44477fe640 1 -- 172.21.15.5:0/1557652052 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16361 (secure 0 0 0) 0x7f444807a770 con 0x7f445013dfd0 2023-11-24T06:16:03.679 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:03.679 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:52.779059+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.678+0000 7f444ffff640 1 -- 172.21.15.5:0/1557652052 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f443006ae40 msgr2=0x7f443006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.678+0000 7f444ffff640 1 --2- 172.21.15.5:0/1557652052 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f443006ae40 0x7f443006d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f44400099b0 tx=0x7f4440008040 comp rx=0 tx=0).stop 2023-11-24T06:16:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.679+0000 7f444ffff640 1 -- 172.21.15.5:0/1557652052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445013dfd0 msgr2=0x7f445015cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.679+0000 7f444ffff640 1 --2- 172.21.15.5:0/1557652052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445013dfd0 0x7f445015cc90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4448009f40 tx=0x7f4448004a40 comp rx=0 tx=0).stop 2023-11-24T06:16:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.679+0000 7f444ffff640 1 -- 172.21.15.5:0/1557652052 shutdown_connections 2023-11-24T06:16:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.679+0000 7f444ffff640 1 --2- 172.21.15.5:0/1557652052 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f443006ae40 0x7f443006d300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.679+0000 7f444ffff640 1 --2- 172.21.15.5:0/1557652052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445013dfd0 0x7f445015cc90 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.679+0000 7f444ffff640 1 --2- 172.21.15.5:0/1557652052 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f44500cf050 0x7f4450156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.679+0000 7f444ffff640 1 --2- 172.21.15.5:0/1557652052 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f445006c130 0x7f44501f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.679+0000 7f444ffff640 1 -- 172.21.15.5:0/1557652052 >> 172.21.15.5:0/1557652052 conn(0x7f445006a860 msgr2=0x7f4450071710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:03.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.679+0000 7f444ffff640 1 -- 172.21.15.5:0/1557652052 shutdown_connections 2023-11-24T06:16:03.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.679+0000 7f444ffff640 1 -- 172.21.15.5:0/1557652052 wait complete. 2023-11-24T06:16:03.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:03.700 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:03.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.921+0000 7fa992e09640 1 Processor -- start 2023-11-24T06:16:03.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.921+0000 7fa992e09640 1 -- start start 2023-11-24T06:16:03.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.922+0000 7fa992e09640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa98c15b350 0x7fa98c0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.922+0000 7fa992e09640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa98c0adb60 0x7fa98c0adf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.922+0000 7fa992e09640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa98c15b730 con 0x7fa98c09cb30 2023-11-24T06:16:03.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.923+0000 7fa992e09640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98c0ac740 con 0x7fa98c0adb60 2023-11-24T06:16:03.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.923+0000 7fa992e09640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98c0ac460 con 0x7fa98c15b350 2023-11-24T06:16:03.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.923+0000 7fa9913a6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa98c0adb60 0x7fa98c0adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.923+0000 7fa9913a6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa98c0adb60 0x7fa98c0adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41508/0 (socket says 172.21.15.5:41508) 2023-11-24T06:16:03.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.923+0000 7fa9913a6640 1 -- 172.21.15.5:0/681418263 learned_addr learned my addr 172.21.15.5:0/681418263 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:03.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.923+0000 7fa990ba5640 1 --2- 172.21.15.5:0/681418263 >> v2:172.21.15.6:3300/0 conn(0x7fa98c15b350 0x7fa98c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.923+0000 7fa9913a6640 1 -- 172.21.15.5:0/681418263 >> v2:172.21.15.6:3300/0 conn(0x7fa98c15b350 msgr2=0x7fa98c0abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:03.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.923+0000 7fa9913a6640 1 --2- 172.21.15.5:0/681418263 >> v2:172.21.15.6:3300/0 conn(0x7fa98c15b350 0x7fa98c0abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.923+0000 7fa9913a6640 1 -- 172.21.15.5:0/681418263 >> v1:172.21.15.5:6789/0 conn(0x7fa98c09cb30 legacy=0x7fa98c0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:03.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.923+0000 7fa98b7fe640 1 -- 172.21.15.5:0/681418263 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714519869 0 0) 0x7fa980001110 con 0x7fa98c09cb30 2023-11-24T06:16:03.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.923+0000 7fa9913a6640 1 -- 172.21.15.5:0/681418263 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c0b2a50 con 0x7fa98c0adb60 2023-11-24T06:16:03.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.924+0000 7fa9913a6640 1 --2- 172.21.15.5:0/681418263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa98c0adb60 0x7fa98c0adf70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0b5100 tx=0x7fa98402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ad54b33874c8cc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:03.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.925+0000 7fa98b7fe640 1 -- 172.21.15.5:0/681418263 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984030860 con 0x7fa98c0adb60 2023-11-24T06:16:03.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.925+0000 7fa98b7fe640 1 -- 172.21.15.5:0/681418263 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9840309c0 con 0x7fa98c0adb60 2023-11-24T06:16:03.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.925+0000 7fa98b7fe640 1 -- 172.21.15.5:0/681418263 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984030d50 con 0x7fa98c0adb60 2023-11-24T06:16:03.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.925+0000 7fa992e09640 1 -- 172.21.15.5:0/681418263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa98c0adb60 msgr2=0x7fa98c0adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:03.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.925+0000 7fa992e09640 1 --2- 172.21.15.5:0/681418263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa98c0adb60 0x7fa98c0adf70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0b5100 tx=0x7fa98402b9a0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.925+0000 7fa992e09640 1 -- 172.21.15.5:0/681418263 shutdown_connections 2023-11-24T06:16:03.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.925+0000 7fa992e09640 1 --2- 172.21.15.5:0/681418263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa98c0adb60 0x7fa98c0adf70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.925+0000 7fa992e09640 1 --2- 172.21.15.5:0/681418263 >> v2:172.21.15.6:3300/0 conn(0x7fa98c15b350 0x7fa98c0abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.925+0000 7fa992e09640 1 -- 172.21.15.5:0/681418263 >> 172.21.15.5:0/681418263 conn(0x7fa98c0b0ba0 msgr2=0x7fa98c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:03.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.926+0000 7fa992e09640 1 -- 172.21.15.5:0/681418263 shutdown_connections 2023-11-24T06:16:03.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.926+0000 7fa992e09640 1 -- 172.21.15.5:0/681418263 wait complete. 2023-11-24T06:16:03.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.927+0000 7fa992e09640 1 Processor -- start 2023-11-24T06:16:03.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.927+0000 7fa992e09640 1 -- start start 2023-11-24T06:16:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.928+0000 7fa992e09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa98c09cb30 0x7fa98c0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.928+0000 7fa992e09640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa98c0adb60 0x7fa98c157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.928+0000 7fa992e09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa98c15b350 0x7fa98c0c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.928+0000 7fa992e09640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c1f2e70 con 0x7fa98c15b350 2023-11-24T06:16:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.928+0000 7fa992e09640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c158b70 con 0x7fa98c0adb60 2023-11-24T06:16:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.928+0000 7fa992e09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98c158e50 con 0x7fa98c09cb30 2023-11-24T06:16:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.929+0000 7fa9913a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa98c15b350 0x7fa98c0c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.929+0000 7fa9913a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa98c15b350 0x7fa98c0c47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41516/0 (socket says 172.21.15.5:41516) 2023-11-24T06:16:03.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.929+0000 7fa9913a6640 1 -- 172.21.15.5:0/3372166823 learned_addr learned my addr 172.21.15.5:0/3372166823 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:03.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.929+0000 7fa990ba5640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa98c09cb30 0x7fa98c0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.929+0000 7fa98bfff640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa98c0adb60 0x7fa98c157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.929+0000 7fa9913a6640 1 -- 172.21.15.5:0/3372166823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa98c09cb30 msgr2=0x7fa98c0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:03.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.929+0000 7fa9913a6640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa98c09cb30 0x7fa98c0afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.929+0000 7fa9913a6640 1 -- 172.21.15.5:0/3372166823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa98c0adb60 msgr2=0x7fa98c157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:03.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.929+0000 7fa9913a6640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa98c0adb60 0x7fa98c157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:03.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.929+0000 7fa9913a6640 1 -- 172.21.15.5:0/3372166823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9840094d0 con 0x7fa98c15b350 2023-11-24T06:16:03.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.929+0000 7fa98bfff640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa98c0adb60 0x7fa98c157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:03.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.929+0000 7fa990ba5640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa98c09cb30 0x7fa98c0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.930+0000 7fa9913a6640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa98c15b350 0x7fa98c0c47f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa9840090b0 tx=0x7fa984004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.930+0000 7fa9897fa640 1 -- 172.21.15.5:0/3372166823 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984039c90 con 0x7fa98c15b350 2023-11-24T06:16:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.930+0000 7fa9897fa640 1 -- 172.21.15.5:0/3372166823 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa984039df0 con 0x7fa98c15b350 2023-11-24T06:16:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.930+0000 7fa9897fa640 1 -- 172.21.15.5:0/3372166823 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984043640 con 0x7fa98c15b350 2023-11-24T06:16:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.930+0000 7fa992e09640 1 -- 172.21.15.5:0/3372166823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98c159130 con 0x7fa98c15b350 2023-11-24T06:16:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.930+0000 7fa992e09640 1 -- 172.21.15.5:0/3372166823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98c0b00a0 con 0x7fa98c15b350 2023-11-24T06:16:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.931+0000 7fa992e09640 1 -- 172.21.15.5:0/3372166823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa98c09d200 con 0x7fa98c15b350 2023-11-24T06:16:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.932+0000 7fa9897fa640 1 -- 172.21.15.5:0/3372166823 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa9840437a0 con 0x7fa98c15b350 2023-11-24T06:16:03.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.933+0000 7fa9897fa640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa97806ae40 0x7fa97806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:03.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.934+0000 7fa9897fa640 1 -- 172.21.15.5:0/3372166823 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa98403e070 con 0x7fa98c15b350 2023-11-24T06:16:03.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.934+0000 7fa990ba5640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa97806ae40 0x7fa97806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:03.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.934+0000 7fa990ba5640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa97806ae40 0x7fa97806d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0aef20 tx=0x7fa97c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:03.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:03.938+0000 7fa9897fa640 1 -- 172.21.15.5:0/3372166823 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa984077820 con 0x7fa98c15b350 2023-11-24T06:16:04.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.139+0000 7fa992e09640 1 -- 172.21.15.5:0/3372166823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa98c06e6f0 con 0x7fa98c15b350 2023-11-24T06:16:04.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.140+0000 7fa9897fa640 1 -- 172.21.15.5:0/3372166823 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa984034020 con 0x7fa98c15b350 2023-11-24T06:16:04.143 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:04.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.143+0000 7fa992e09640 1 -- 172.21.15.5:0/3372166823 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa97806ae40 msgr2=0x7fa97806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:04.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.143+0000 7fa992e09640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa97806ae40 0x7fa97806d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0aef20 tx=0x7fa97c00b040 comp rx=0 tx=0).stop 2023-11-24T06:16:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.143+0000 7fa992e09640 1 -- 172.21.15.5:0/3372166823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa98c15b350 msgr2=0x7fa98c0c47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.143+0000 7fa992e09640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa98c15b350 0x7fa98c0c47f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa9840090b0 tx=0x7fa984004730 comp rx=0 tx=0).stop 2023-11-24T06:16:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.143+0000 7fa992e09640 1 -- 172.21.15.5:0/3372166823 shutdown_connections 2023-11-24T06:16:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.143+0000 7fa992e09640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa97806ae40 0x7fa97806d300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.143+0000 7fa992e09640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa98c15b350 0x7fa98c0c47f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.143+0000 7fa992e09640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa98c0adb60 0x7fa98c157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.143+0000 7fa992e09640 1 --2- 172.21.15.5:0/3372166823 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa98c09cb30 0x7fa98c0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.143+0000 7fa992e09640 1 -- 172.21.15.5:0/3372166823 >> 172.21.15.5:0/3372166823 conn(0x7fa98c0b0ba0 msgr2=0x7fa98c159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.143+0000 7fa992e09640 1 -- 172.21.15.5:0/3372166823 shutdown_connections 2023-11-24T06:16:04.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.144+0000 7fa992e09640 1 -- 172.21.15.5:0/3372166823 wait complete. 2023-11-24T06:16:04.166 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:04.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.390+0000 7f45a6c31640 1 Processor -- start 2023-11-24T06:16:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.391+0000 7f45a6c31640 1 -- start start 2023-11-24T06:16:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.392+0000 7f45a6c31640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45a009cb30 0x7f45a00abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.392+0000 7f45a6c31640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f45a00adb60 0x7f45a00adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.392+0000 7f45a6c31640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a015b730 con 0x7f45a015b350 2023-11-24T06:16:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.392+0000 7f45a6c31640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a00ac740 con 0x7f45a00adb60 2023-11-24T06:16:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.392+0000 7f45a6c31640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00ac460 con 0x7f45a009cb30 2023-11-24T06:16:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.393+0000 7f45a49cd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f45a015b350 0x7f45a00b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37092/0 (socket says 172.21.15.5:37092) 2023-11-24T06:16:04.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.393+0000 7f45a49cd640 1 -- 172.21.15.5:0/3411084942 learned_addr learned my addr 172.21.15.5:0/3411084942 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.393+0000 7f459f7fe640 1 -- 172.21.15.5:0/3411084942 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190150097 0 0) 0x7f4594001110 con 0x7f45a015b350 2023-11-24T06:16:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.393+0000 7f459ffff640 1 --2- 172.21.15.5:0/3411084942 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45a009cb30 0x7f45a00abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.393+0000 7f45a51ce640 1 --2- 172.21.15.5:0/3411084942 >> v2:172.21.15.6:3300/0 conn(0x7f45a00adb60 0x7f45a00adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.393+0000 7f459f7fe640 1 -- 172.21.15.5:0/3411084942 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4590003520 con 0x7f45a015b350 2023-11-24T06:16:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459f7fe640 1 -- 172.21.15.5:0/3411084942 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 83457832 0 0) 0x7f4590003520 con 0x7f45a015b350 2023-11-24T06:16:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459f7fe640 1 -- 172.21.15.5:0/3411084942 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45a009cb30 msgr2=0x7f45a00abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459f7fe640 1 --2- 172.21.15.5:0/3411084942 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45a009cb30 0x7f45a00abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459f7fe640 1 -- 172.21.15.5:0/3411084942 >> v2:172.21.15.6:3300/0 conn(0x7f45a00adb60 msgr2=0x7f45a00adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459f7fe640 1 --2- 172.21.15.5:0/3411084942 >> v2:172.21.15.6:3300/0 conn(0x7f45a00adb60 0x7f45a00adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459f7fe640 1 -- 172.21.15.5:0/3411084942 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a00b2a50 con 0x7f45a015b350 2023-11-24T06:16:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459f7fe640 1 -- 172.21.15.5:0/3411084942 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f4594001800 con 0x7f45a015b350 2023-11-24T06:16:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459f7fe640 1 -- 172.21.15.5:0/3411084942 >> v1:172.21.15.5:6789/0 conn(0x7f45a015b350 legacy=0x7f45a00b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459f7fe640 1 --2- 172.21.15.5:0/3411084942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4590003970 0x7f4590003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459f7fe640 1 --2- 172.21.15.5:0/3411084942 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45900047c0 0x7f4590004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459f7fe640 1 --2- 172.21.15.5:0/3411084942 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4590005180 0x7f459000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459f7fe640 1 -- 172.21.15.5:0/3411084942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00ac740 con 0x7f4590003970 2023-11-24T06:16:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459f7fe640 1 -- 172.21.15.5:0/3411084942 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00ac460 con 0x7f45900047c0 2023-11-24T06:16:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459f7fe640 1 -- 172.21.15.5:0/3411084942 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459000bf70 con 0x7f4590005180 2023-11-24T06:16:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f459ffff640 1 --2- 172.21.15.5:0/3411084942 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45a009cb30 0x7f45a00abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:16:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f45a49cd640 1 --2- 172.21.15.5:0/3411084942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4590003970 0x7f4590003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f45a51ce640 1 --2- 172.21.15.5:0/3411084942 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4590005180 0x7f459000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f45a49cd640 1 -- 172.21.15.5:0/3411084942 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4590005180 msgr2=0x7f459000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f45a49cd640 1 --2- 172.21.15.5:0/3411084942 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4590005180 0x7f459000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f45a49cd640 1 -- 172.21.15.5:0/3411084942 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45900047c0 msgr2=0x7f4590004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f45a49cd640 1 --2- 172.21.15.5:0/3411084942 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45900047c0 0x7f4590004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.394+0000 7f45a49cd640 1 -- 172.21.15.5:0/3411084942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4590003520 con 0x7f4590003970 2023-11-24T06:16:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.395+0000 7f45a49cd640 1 --2- 172.21.15.5:0/3411084942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4590003970 0x7f4590003d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f459400c110 tx=0x7f45900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b24c9ea0c0406b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.396+0000 7f459f7fe640 1 -- 172.21.15.5:0/3411084942 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594005ad0 con 0x7f4590003970 2023-11-24T06:16:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.396+0000 7f459f7fe640 1 -- 172.21.15.5:0/3411084942 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4594005c50 con 0x7f4590003970 2023-11-24T06:16:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.396+0000 7f459f7fe640 1 -- 172.21.15.5:0/3411084942 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459402c810 con 0x7f4590003970 2023-11-24T06:16:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.396+0000 7f45a6c31640 1 -- 172.21.15.5:0/3411084942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4590003970 msgr2=0x7f4590003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.396+0000 7f45a6c31640 1 --2- 172.21.15.5:0/3411084942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4590003970 0x7f4590003d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f459400c110 tx=0x7f45900057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.396+0000 7f45a51ce640 1 -- 172.21.15.5:0/3411084942 reap_dead start 2023-11-24T06:16:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.396+0000 7f45a6c31640 1 -- 172.21.15.5:0/3411084942 shutdown_connections 2023-11-24T06:16:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.396+0000 7f45a6c31640 1 -- 172.21.15.5:0/3411084942 >> 172.21.15.5:0/3411084942 conn(0x7f45a00b0ba0 msgr2=0x7f45a00b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.396+0000 7f45a6c31640 1 -- 172.21.15.5:0/3411084942 shutdown_connections 2023-11-24T06:16:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.396+0000 7f45a6c31640 1 -- 172.21.15.5:0/3411084942 wait complete. 2023-11-24T06:16:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.398+0000 7f45a6c31640 1 Processor -- start 2023-11-24T06:16:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.398+0000 7f45a6c31640 1 -- start start 2023-11-24T06:16:04.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.399+0000 7f45a6c31640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4590003970 0x7f45a015cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.399+0000 7f45a6c31640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45a015b8d0 0x7f45a01584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.399+0000 7f45a6c31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a015e3c0 0x7f45a013e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.399+0000 7f45a6c31640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a015be00 con 0x7f45a015e3c0 2023-11-24T06:16:04.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.399+0000 7f45a6c31640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a015c0e0 con 0x7f45a015b8d0 2023-11-24T06:16:04.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.399+0000 7f45a6c31640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a015c3c0 con 0x7f4590003970 2023-11-24T06:16:04.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.400+0000 7f45a51ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a015e3c0 0x7f45a013e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.400+0000 7f45a51ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a015e3c0 0x7f45a013e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41532/0 (socket says 172.21.15.5:41532) 2023-11-24T06:16:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.400+0000 7f45a51ce640 1 -- 172.21.15.5:0/3413841122 learned_addr learned my addr 172.21.15.5:0/3413841122 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.400+0000 7f45a49cd640 1 --2- 172.21.15.5:0/3413841122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4590003970 0x7f45a015cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.400+0000 7f459ffff640 1 --2- 172.21.15.5:0/3413841122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45a015b8d0 0x7f45a01584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.400+0000 7f45a51ce640 1 -- 172.21.15.5:0/3413841122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4590003970 msgr2=0x7f45a015cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.400+0000 7f45a51ce640 1 --2- 172.21.15.5:0/3413841122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4590003970 0x7f45a015cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.400+0000 7f45a51ce640 1 -- 172.21.15.5:0/3413841122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45a015b8d0 msgr2=0x7f45a01584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.400+0000 7f45a51ce640 1 --2- 172.21.15.5:0/3413841122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45a015b8d0 0x7f45a01584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.400+0000 7f45a51ce640 1 -- 172.21.15.5:0/3413841122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a015c6a0 con 0x7f45a015e3c0 2023-11-24T06:16:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.400+0000 7f45a51ce640 1 --2- 172.21.15.5:0/3413841122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a015e3c0 0x7f45a013e610 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f45a0067e90 tx=0x7f459800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.401+0000 7f459d7fa640 1 -- 172.21.15.5:0/3413841122 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598016620 con 0x7f45a015e3c0 2023-11-24T06:16:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.401+0000 7f459d7fa640 1 -- 172.21.15.5:0/3413841122 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45980167a0 con 0x7f45a015e3c0 2023-11-24T06:16:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.401+0000 7f459d7fa640 1 -- 172.21.15.5:0/3413841122 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4598016b50 con 0x7f45a015e3c0 2023-11-24T06:16:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.401+0000 7f45a6c31640 1 -- 172.21.15.5:0/3413841122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a015c980 con 0x7f45a015e3c0 2023-11-24T06:16:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.401+0000 7f45a6c31640 1 -- 172.21.15.5:0/3413841122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a015e840 con 0x7f45a015e3c0 2023-11-24T06:16:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.402+0000 7f45a6c31640 1 -- 172.21.15.5:0/3413841122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a006e6d0 con 0x7f45a015e3c0 2023-11-24T06:16:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.403+0000 7f459d7fa640 1 -- 172.21.15.5:0/3413841122 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4598016cb0 con 0x7f45a015e3c0 2023-11-24T06:16:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.404+0000 7f459d7fa640 1 --2- 172.21.15.5:0/3413841122 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f458406ae70 0x7f458406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.404+0000 7f459d7fa640 1 -- 172.21.15.5:0/3413841122 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f459808a2b0 con 0x7f45a015e3c0 2023-11-24T06:16:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.404+0000 7f45a49cd640 1 --2- 172.21.15.5:0/3413841122 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f458406ae70 0x7f458406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.405+0000 7f45a49cd640 1 --2- 172.21.15.5:0/3413841122 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f458406ae70 0x7f458406d330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4594034000 tx=0x7f4594007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:04.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.409+0000 7f459d7fa640 1 -- 172.21.15.5:0/3413841122 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4598055360 con 0x7f45a015e3c0 2023-11-24T06:16:04.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.605+0000 7f45a6c31640 1 -- 172.21.15.5:0/3413841122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45a009d200 con 0x7f45a015e3c0 2023-11-24T06:16:04.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.606+0000 7f459d7fa640 1 -- 172.21.15.5:0/3413841122 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4598058980 con 0x7f45a015e3c0 2023-11-24T06:16:04.609 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.609+0000 7f45a6c31640 1 -- 172.21.15.5:0/3413841122 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f458406ae70 msgr2=0x7f458406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.609+0000 7f45a6c31640 1 --2- 172.21.15.5:0/3413841122 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f458406ae70 0x7f458406d330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4594034000 tx=0x7f4594007000 comp rx=0 tx=0).stop 2023-11-24T06:16:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.609+0000 7f45a6c31640 1 -- 172.21.15.5:0/3413841122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a015e3c0 msgr2=0x7f45a013e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.609+0000 7f45a6c31640 1 --2- 172.21.15.5:0/3413841122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a015e3c0 0x7f45a013e610 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f45a0067e90 tx=0x7f459800f520 comp rx=0 tx=0).stop 2023-11-24T06:16:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.610+0000 7f45a6c31640 1 -- 172.21.15.5:0/3413841122 shutdown_connections 2023-11-24T06:16:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.610+0000 7f45a6c31640 1 --2- 172.21.15.5:0/3413841122 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f458406ae70 0x7f458406d330 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.610+0000 7f45a6c31640 1 --2- 172.21.15.5:0/3413841122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a015e3c0 0x7f45a013e610 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.610+0000 7f45a6c31640 1 --2- 172.21.15.5:0/3413841122 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45a015b8d0 0x7f45a01584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.610+0000 7f45a6c31640 1 --2- 172.21.15.5:0/3413841122 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4590003970 0x7f45a015cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.610+0000 7f45a6c31640 1 -- 172.21.15.5:0/3413841122 >> 172.21.15.5:0/3413841122 conn(0x7f45a00b0ba0 msgr2=0x7f45a00b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.610+0000 7f45a6c31640 1 -- 172.21.15.5:0/3413841122 shutdown_connections 2023-11-24T06:16:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.610+0000 7f45a6c31640 1 -- 172.21.15.5:0/3413841122 wait complete. 2023-11-24T06:16:04.630 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.856+0000 7f2ac3432640 1 Processor -- start 2023-11-24T06:16:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.856+0000 7f2ac3432640 1 -- start start 2023-11-24T06:16:04.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.857+0000 7f2ac3432640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2abc15d3b0 0x7f2abc0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.857+0000 7f2ac3432640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2abc0d1d90 0x7f2abc0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.857+0000 7f2ac3432640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2abc0c6bf0 con 0x7f2abc0d0fa0 2023-11-24T06:16:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.857+0000 7f2ac3432640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2abc0d1400 con 0x7f2abc0d1d90 2023-11-24T06:16:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.857+0000 7f2ac3432640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc0cedc0 con 0x7f2abc15d3b0 2023-11-24T06:16:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.858+0000 7f2ac1c2f640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2abc0d0fa0 0x7f2abc0d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37108/0 (socket says 172.21.15.5:37108) 2023-11-24T06:16:04.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.858+0000 7f2ac1c2f640 1 -- 172.21.15.5:0/2070372631 learned_addr learned my addr 172.21.15.5:0/2070372631 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:04.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.858+0000 7f2ac2430640 1 --2- 172.21.15.5:0/2070372631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2abc15d3b0 0x7f2abc0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.858+0000 7f2ac2c31640 1 --2- 172.21.15.5:0/2070372631 >> v2:172.21.15.6:3300/0 conn(0x7f2abc0d1d90 0x7f2abc0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.858+0000 7f2ac142e640 1 -- 172.21.15.5:0/2070372631 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296885331 0 0) 0x7f2abc0c6bf0 con 0x7f2abc0d0fa0 2023-11-24T06:16:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 -- 172.21.15.5:0/2070372631 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2aa8003520 con 0x7f2abc0d0fa0 2023-11-24T06:16:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 -- 172.21.15.5:0/2070372631 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1358850694 0 0) 0x7f2aa8003520 con 0x7f2abc0d0fa0 2023-11-24T06:16:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 -- 172.21.15.5:0/2070372631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2abc15d3b0 msgr2=0x7f2abc0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 --2- 172.21.15.5:0/2070372631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2abc15d3b0 0x7f2abc0ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 -- 172.21.15.5:0/2070372631 >> v2:172.21.15.6:3300/0 conn(0x7f2abc0d1d90 msgr2=0x7f2abc0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 --2- 172.21.15.5:0/2070372631 >> v2:172.21.15.6:3300/0 conn(0x7f2abc0d1d90 0x7f2abc0b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 -- 172.21.15.5:0/2070372631 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc0c6e40 con 0x7f2abc0d0fa0 2023-11-24T06:16:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 -- 172.21.15.5:0/2070372631 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f2ab0001670 con 0x7f2abc0d0fa0 2023-11-24T06:16:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac2430640 1 --2- 172.21.15.5:0/2070372631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2abc15d3b0 0x7f2abc0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:16:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 -- 172.21.15.5:0/2070372631 >> v1:172.21.15.5:6789/0 conn(0x7f2abc0d0fa0 legacy=0x7f2abc0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 --2- 172.21.15.5:0/2070372631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2aa80039e0 0x7f2aa8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 --2- 172.21.15.5:0/2070372631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2aa80047c0 0x7f2aa8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 --2- 172.21.15.5:0/2070372631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8005180 0x7f2aa800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 -- 172.21.15.5:0/2070372631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc0d1400 con 0x7f2aa8005180 2023-11-24T06:16:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 -- 172.21.15.5:0/2070372631 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc0cedc0 con 0x7f2aa80047c0 2023-11-24T06:16:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac142e640 1 -- 172.21.15.5:0/2070372631 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa800bf70 con 0x7f2aa80039e0 2023-11-24T06:16:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.859+0000 7f2ac2c31640 1 --2- 172.21.15.5:0/2070372631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8005180 0x7f2aa800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.860+0000 7f2ac2430640 1 --2- 172.21.15.5:0/2070372631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2aa80039e0 0x7f2aa8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.860+0000 7f2ac1c2f640 1 --2- 172.21.15.5:0/2070372631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2aa80047c0 0x7f2aa8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.860+0000 7f2ac2c31640 1 -- 172.21.15.5:0/2070372631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2aa80039e0 msgr2=0x7f2aa8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.860+0000 7f2ac2c31640 1 --2- 172.21.15.5:0/2070372631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2aa80039e0 0x7f2aa8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.860+0000 7f2ac2c31640 1 -- 172.21.15.5:0/2070372631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2aa80047c0 msgr2=0x7f2aa8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.860+0000 7f2ac2c31640 1 --2- 172.21.15.5:0/2070372631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2aa80047c0 0x7f2aa8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.860+0000 7f2ac2c31640 1 -- 172.21.15.5:0/2070372631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa8003520 con 0x7f2aa8005180 2023-11-24T06:16:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.860+0000 7f2ac1c2f640 1 --2- 172.21.15.5:0/2070372631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2aa80047c0 0x7f2aa8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.861+0000 7f2ac2c31640 1 --2- 172.21.15.5:0/2070372631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8005180 0x7f2aa800b980 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009f90 tx=0x7f2aa80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613e25f0e962b502 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.861+0000 7f2ac142e640 1 -- 172.21.15.5:0/2070372631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab402cc20 con 0x7f2aa8005180 2023-11-24T06:16:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.861+0000 7f2ac142e640 1 -- 172.21.15.5:0/2070372631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab400f040 con 0x7f2aa8005180 2023-11-24T06:16:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.861+0000 7f2ac142e640 1 -- 172.21.15.5:0/2070372631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4034440 con 0x7f2aa8005180 2023-11-24T06:16:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.862+0000 7f2ac3432640 1 -- 172.21.15.5:0/2070372631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8005180 msgr2=0x7f2aa800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.862+0000 7f2ac3432640 1 --2- 172.21.15.5:0/2070372631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa8005180 0x7f2aa800b980 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009f90 tx=0x7f2aa80057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.862+0000 7f2ac2c31640 1 -- 172.21.15.5:0/2070372631 reap_dead start 2023-11-24T06:16:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.862+0000 7f2ac3432640 1 -- 172.21.15.5:0/2070372631 shutdown_connections 2023-11-24T06:16:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.862+0000 7f2ac3432640 1 -- 172.21.15.5:0/2070372631 >> 172.21.15.5:0/2070372631 conn(0x7f2abc0d30a0 msgr2=0x7f2abc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.862+0000 7f2ac3432640 1 -- 172.21.15.5:0/2070372631 shutdown_connections 2023-11-24T06:16:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.862+0000 7f2ac3432640 1 -- 172.21.15.5:0/2070372631 wait complete. 2023-11-24T06:16:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.864+0000 7f2ac3432640 1 Processor -- start 2023-11-24T06:16:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.864+0000 7f2ac3432640 1 -- start start 2023-11-24T06:16:04.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.865+0000 7f2ac3432640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2aa8005180 0x7f2abc09feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.865+0000 7f2ac3432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2abc0a03f0 0x7f2abc0b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.865+0000 7f2ac3432640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc0b7180 0x7f2abc06af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.865+0000 7f2ac3432640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc0b7600 con 0x7f2abc0a03f0 2023-11-24T06:16:04.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.865+0000 7f2ac3432640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc09e390 con 0x7f2abc0b7180 2023-11-24T06:16:04.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.865+0000 7f2ac3432640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2abc15aac0 con 0x7f2aa8005180 2023-11-24T06:16:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.866+0000 7f2ac1c2f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2abc0a03f0 0x7f2abc0b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.866+0000 7f2ac1c2f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2abc0a03f0 0x7f2abc0b6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41548/0 (socket says 172.21.15.5:41548) 2023-11-24T06:16:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.866+0000 7f2ac1c2f640 1 -- 172.21.15.5:0/761240905 learned_addr learned my addr 172.21.15.5:0/761240905 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.866+0000 7f2ac2430640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2aa8005180 0x7f2abc09feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.866+0000 7f2ac2c31640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc0b7180 0x7f2abc06af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.866+0000 7f2ac1c2f640 1 -- 172.21.15.5:0/761240905 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2aa8005180 msgr2=0x7f2abc09feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.866+0000 7f2ac1c2f640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2aa8005180 0x7f2abc09feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.866+0000 7f2ac1c2f640 1 -- 172.21.15.5:0/761240905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc0b7180 msgr2=0x7f2abc06af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.866+0000 7f2ac1c2f640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc0b7180 0x7f2abc06af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.866+0000 7f2ac1c2f640 1 -- 172.21.15.5:0/761240905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2abc09ee10 con 0x7f2abc0a03f0 2023-11-24T06:16:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.866+0000 7f2ac2c31640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc0b7180 0x7f2abc06af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.866+0000 7f2ac1c2f640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2abc0a03f0 0x7f2abc0b6c40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000d890 tx=0x7f2ab000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.867+0000 7f2aaeffd640 1 -- 172.21.15.5:0/761240905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0007560 con 0x7f2abc0a03f0 2023-11-24T06:16:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.867+0000 7f2aaeffd640 1 -- 172.21.15.5:0/761240905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab00076c0 con 0x7f2abc0a03f0 2023-11-24T06:16:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.867+0000 7f2aaeffd640 1 -- 172.21.15.5:0/761240905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab00049e0 con 0x7f2abc0a03f0 2023-11-24T06:16:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.867+0000 7f2ac3432640 1 -- 172.21.15.5:0/761240905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2abc09f0f0 con 0x7f2abc0a03f0 2023-11-24T06:16:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.867+0000 7f2ac3432640 1 -- 172.21.15.5:0/761240905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2abc09f6b0 con 0x7f2abc0a03f0 2023-11-24T06:16:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.868+0000 7f2ac3432640 1 -- 172.21.15.5:0/761240905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2abc0d0bd0 con 0x7f2abc0a03f0 2023-11-24T06:16:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.869+0000 7f2aaeffd640 1 -- 172.21.15.5:0/761240905 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ab0004b40 con 0x7f2abc0a03f0 2023-11-24T06:16:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.870+0000 7f2aaeffd640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2aa006ae40 0x7f2aa006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.870+0000 7f2aaeffd640 1 -- 172.21.15.5:0/761240905 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ab0088ec0 con 0x7f2abc0a03f0 2023-11-24T06:16:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.870+0000 7f2ac2430640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2aa006ae40 0x7f2aa006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.871+0000 7f2ac2430640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2aa006ae40 0x7f2aa006d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2abc08b2a0 tx=0x7f2ab8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:04.874+0000 7f2aaeffd640 1 -- 172.21.15.5:0/761240905 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ab004de90 con 0x7f2abc0a03f0 2023-11-24T06:16:05.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.069+0000 7f2ac3432640 1 -- 172.21.15.5:0/761240905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2abc04e840 con 0x7f2abc0a03f0 2023-11-24T06:16:05.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.069+0000 7f2aaeffd640 1 -- 172.21.15.5:0/761240905 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2ab00190a0 con 0x7f2abc0a03f0 2023-11-24T06:16:05.072 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:05.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.072+0000 7f2ac3432640 1 -- 172.21.15.5:0/761240905 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2aa006ae40 msgr2=0x7f2aa006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:05.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.072+0000 7f2ac3432640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2aa006ae40 0x7f2aa006d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2abc08b2a0 tx=0x7f2ab8008040 comp rx=0 tx=0).stop 2023-11-24T06:16:05.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.072+0000 7f2ac3432640 1 -- 172.21.15.5:0/761240905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2abc0a03f0 msgr2=0x7f2abc0b6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:05.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.072+0000 7f2ac3432640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2abc0a03f0 0x7f2abc0b6c40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000d890 tx=0x7f2ab000dd50 comp rx=0 tx=0).stop 2023-11-24T06:16:05.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.072+0000 7f2ac3432640 1 -- 172.21.15.5:0/761240905 shutdown_connections 2023-11-24T06:16:05.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.072+0000 7f2ac3432640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2aa006ae40 0x7f2aa006d300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.072+0000 7f2ac3432640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2abc0b7180 0x7f2abc06af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.072+0000 7f2ac3432640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2abc0a03f0 0x7f2abc0b6c40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.072+0000 7f2ac3432640 1 --2- 172.21.15.5:0/761240905 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2aa8005180 0x7f2abc09feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.072+0000 7f2ac3432640 1 -- 172.21.15.5:0/761240905 >> 172.21.15.5:0/761240905 conn(0x7f2abc0d30a0 msgr2=0x7f2abc0b5370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:05.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.073+0000 7f2ac3432640 1 -- 172.21.15.5:0/761240905 shutdown_connections 2023-11-24T06:16:05.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.073+0000 7f2ac3432640 1 -- 172.21.15.5:0/761240905 wait complete. 2023-11-24T06:16:05.092 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:16:05.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.296+0000 7fca87fff640 1 Processor -- start 2023-11-24T06:16:05.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.296+0000 7fca87fff640 1 -- start start 2023-11-24T06:16:05.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.298+0000 7fca87fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fca8806d250 0x7fca8815d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:05.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.298+0000 7fca87fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fca8815ddb0 0x7fca88070720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:05.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.298+0000 7fca87fff640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca88161670 con 0x7fca88157770 2023-11-24T06:16:05.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.298+0000 7fca87fff640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca8815e610 con 0x7fca8815ddb0 2023-11-24T06:16:05.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.298+0000 7fca87fff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca88160d40 con 0x7fca8806d250 2023-11-24T06:16:05.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.299+0000 7fca877fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fca8815ddb0 0x7fca88070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:05.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.299+0000 7fca877fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fca8815ddb0 0x7fca88070720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41560/0 (socket says 172.21.15.5:41560) 2023-11-24T06:16:05.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.299+0000 7fca877fe640 1 -- 172.21.15.5:0/3221760768 learned_addr learned my addr 172.21.15.5:0/3221760768 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:05.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.299+0000 7fca7e7fc640 1 --2- 172.21.15.5:0/3221760768 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fca8806d250 0x7fca8815d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:05.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.299+0000 7fca877fe640 1 -- 172.21.15.5:0/3221760768 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fca8806d250 msgr2=0x7fca8815d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:05.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.299+0000 7fca877fe640 1 --2- 172.21.15.5:0/3221760768 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fca8806d250 0x7fca8815d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.299+0000 7fca877fe640 1 -- 172.21.15.5:0/3221760768 >> v1:172.21.15.6:6789/0 conn(0x7fca88157770 legacy=0x7fca8806cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:05.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.299+0000 7fca877fe640 1 -- 172.21.15.5:0/3221760768 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca88161020 con 0x7fca8815ddb0 2023-11-24T06:16:05.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.300+0000 7fca877fe640 1 --2- 172.21.15.5:0/3221760768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca8815ddb0 0x7fca88070720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fca80009e50 tx=0x7fca8002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe890c173503e15 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:05.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.301+0000 7fca867fc640 1 -- 172.21.15.5:0/3221760768 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8002ca70 con 0x7fca8815ddb0 2023-11-24T06:16:05.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.301+0000 7fca867fc640 1 -- 172.21.15.5:0/3221760768 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8002cbd0 con 0x7fca8815ddb0 2023-11-24T06:16:05.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.301+0000 7fca867fc640 1 -- 172.21.15.5:0/3221760768 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca800343f0 con 0x7fca8815ddb0 2023-11-24T06:16:05.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.301+0000 7fca87fff640 1 -- 172.21.15.5:0/3221760768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca8815ddb0 msgr2=0x7fca88070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:05.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.301+0000 7fca87fff640 1 --2- 172.21.15.5:0/3221760768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca8815ddb0 0x7fca88070720 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fca80009e50 tx=0x7fca8002bed0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.301+0000 7fca87fff640 1 -- 172.21.15.5:0/3221760768 shutdown_connections 2023-11-24T06:16:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.301+0000 7fca87fff640 1 --2- 172.21.15.5:0/3221760768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca8815ddb0 0x7fca88070720 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.301+0000 7fca87fff640 1 --2- 172.21.15.5:0/3221760768 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fca8806d250 0x7fca8815d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.301+0000 7fca87fff640 1 -- 172.21.15.5:0/3221760768 >> 172.21.15.5:0/3221760768 conn(0x7fca8806a860 msgr2=0x7fca8806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.302+0000 7fca87fff640 1 -- 172.21.15.5:0/3221760768 shutdown_connections 2023-11-24T06:16:05.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.302+0000 7fca87fff640 1 -- 172.21.15.5:0/3221760768 wait complete. 2023-11-24T06:16:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.303+0000 7fca87fff640 1 Processor -- start 2023-11-24T06:16:05.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.303+0000 7fca87fff640 1 -- start start 2023-11-24T06:16:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.304+0000 7fca87fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fca8806d250 0x7fca8814b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.304+0000 7fca87fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca88157770 0x7fca8814a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.304+0000 7fca87fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8815ddb0 0x7fca881f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.304+0000 7fca87fff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8815f9b0 con 0x7fca88157770 2023-11-24T06:16:05.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.304+0000 7fca87fff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8815fee0 con 0x7fca8815ddb0 2023-11-24T06:16:05.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.304+0000 7fca87fff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca881f7070 con 0x7fca8806d250 2023-11-24T06:16:05.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.305+0000 7fca7e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca88157770 0x7fca8814a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:05.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.305+0000 7fca7e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca88157770 0x7fca8814a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41566/0 (socket says 172.21.15.5:41566) 2023-11-24T06:16:05.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.305+0000 7fca7e7fc640 1 -- 172.21.15.5:0/2303592463 learned_addr learned my addr 172.21.15.5:0/2303592463 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:05.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.305+0000 7fca877fe640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8815ddb0 0x7fca881f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:05.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.305+0000 7fca7e7fc640 1 -- 172.21.15.5:0/2303592463 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fca8806d250 msgr2=0x7fca8814b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.305+0000 7fca86ffd640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fca8806d250 0x7fca8814b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.305+0000 7fca7e7fc640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fca8806d250 0x7fca8814b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.305+0000 7fca7e7fc640 1 -- 172.21.15.5:0/2303592463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8815ddb0 msgr2=0x7fca881f62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.305+0000 7fca7e7fc640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8815ddb0 0x7fca881f62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.305+0000 7fca7e7fc640 1 -- 172.21.15.5:0/2303592463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca80009b30 con 0x7fca88157770 2023-11-24T06:16:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.305+0000 7fca86ffd640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fca8806d250 0x7fca8814b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.305+0000 7fca877fe640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8815ddb0 0x7fca881f62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:16:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.306+0000 7fca7e7fc640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca88157770 0x7fca8814a780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca7000d800 tx=0x7fca7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.306+0000 7fca7ffff640 1 -- 172.21.15.5:0/2303592463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7000b6c0 con 0x7fca88157770 2023-11-24T06:16:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.306+0000 7fca7ffff640 1 -- 172.21.15.5:0/2303592463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7000b820 con 0x7fca88157770 2023-11-24T06:16:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.306+0000 7fca7ffff640 1 -- 172.21.15.5:0/2303592463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7000bbb0 con 0x7fca88157770 2023-11-24T06:16:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.306+0000 7fca87fff640 1 -- 172.21.15.5:0/2303592463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca88161c80 con 0x7fca88157770 2023-11-24T06:16:05.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.306+0000 7fca87fff640 1 -- 172.21.15.5:0/2303592463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca880caf40 con 0x7fca88157770 2023-11-24T06:16:05.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.307+0000 7fca87fff640 1 -- 172.21.15.5:0/2303592463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8804e840 con 0x7fca88157770 2023-11-24T06:16:05.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.308+0000 7fca7ffff640 1 -- 172.21.15.5:0/2303592463 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fca7000bd10 con 0x7fca88157770 2023-11-24T06:16:05.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.309+0000 7fca7ffff640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fca6806ae40 0x7fca6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:05.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.309+0000 7fca7ffff640 1 -- 172.21.15.5:0/2303592463 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca70089bc0 con 0x7fca88157770 2023-11-24T06:16:05.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.309+0000 7fca86ffd640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fca6806ae40 0x7fca6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:05.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.310+0000 7fca86ffd640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fca6806ae40 0x7fca6806d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fca8815f2d0 tx=0x7fca78009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:05.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.313+0000 7fca7ffff640 1 -- 172.21.15.5:0/2303592463 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fca70054c60 con 0x7fca88157770 2023-11-24T06:16:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.536+0000 7fca87fff640 1 -- 172.21.15.5:0/2303592463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca88161ae0 con 0x7fca88157770 2023-11-24T06:16:05.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.537+0000 7fca7ffff640 1 -- 172.21.15.5:0/2303592463 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4895 (secure 0 0 0) 0x7fca88161ae0 con 0x7fca88157770 2023-11-24T06:16:05.540 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:05.540 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4009460946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4009460946},{"type":"v1","addr":"172.21.15.5:6837","nonce":4009460946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2496926427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2496926427},{"type":"v1","addr":"172.21.15.5:6836","nonce":2496926427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3829754769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3829754769},{"type":"v1","addr":"172.21.15.6:6835","nonce":3829754769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:15:51.611541+0000","modified":"2023-11-24T06:15:54.664234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6833/2760317954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2760317954},{"type":"v1","addr":"172.21.15.6:6833","nonce":2760317954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:16:05.541 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2023-11-24T06:16:05.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.539+0000 7fca87fff640 1 -- 172.21.15.5:0/2303592463 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fca6806ae40 msgr2=0x7fca6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.539+0000 7fca87fff640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fca6806ae40 0x7fca6806d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fca8815f2d0 tx=0x7fca78009040 comp rx=0 tx=0).stop 2023-11-24T06:16:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.539+0000 7fca87fff640 1 -- 172.21.15.5:0/2303592463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca88157770 msgr2=0x7fca8814a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.539+0000 7fca87fff640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca88157770 0x7fca8814a780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fca7000d800 tx=0x7fca7000dcc0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.540+0000 7fca87fff640 1 -- 172.21.15.5:0/2303592463 shutdown_connections 2023-11-24T06:16:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.540+0000 7fca87fff640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fca6806ae40 0x7fca6806d300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.540+0000 7fca87fff640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fca8815ddb0 0x7fca881f62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.540+0000 7fca87fff640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca88157770 0x7fca8814a780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.540+0000 7fca87fff640 1 --2- 172.21.15.5:0/2303592463 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fca8806d250 0x7fca8814b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.540+0000 7fca87fff640 1 -- 172.21.15.5:0/2303592463 >> 172.21.15.5:0/2303592463 conn(0x7fca8806a860 msgr2=0x7fca880cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.540+0000 7fca87fff640 1 -- 172.21.15.5:0/2303592463 shutdown_connections 2023-11-24T06:16:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.540+0000 7fca87fff640 1 -- 172.21.15.5:0/2303592463 wait complete. 2023-11-24T06:16:05.560 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.779+0000 7fb50e38f640 1 Processor -- start 2023-11-24T06:16:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.779+0000 7fb50e38f640 1 -- start start 2023-11-24T06:16:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.781+0000 7fb50e38f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb50813dfd0 0x7fb50815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.781+0000 7fb50e38f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5080cf050 0x7fb5080cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.781+0000 7fb50e38f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb50806c130 0x7fb50806c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.781+0000 7fb50e38f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5080cfa10 con 0x7fb50813dfd0 2023-11-24T06:16:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.781+0000 7fb50e38f640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb50815fa70 con 0x7fb50806c130 2023-11-24T06:16:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.781+0000 7fb50e38f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50815ee60 con 0x7fb5080cf050 2023-11-24T06:16:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.781+0000 7fb50d38d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb50813dfd0 0x7fb50815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.781+0000 7fb50d38d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb50813dfd0 0x7fb50815be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41574/0 (socket says 172.21.15.5:41574) 2023-11-24T06:16:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.781+0000 7fb50d38d640 1 -- 172.21.15.5:0/2733125314 learned_addr learned my addr 172.21.15.5:0/2733125314 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:05.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.782+0000 7fb50db8e640 1 --2- 172.21.15.5:0/2733125314 >> v2:172.21.15.6:3300/0 conn(0x7fb50806c130 0x7fb50806c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.782+0000 7fb50cb8c640 1 --2- 172.21.15.5:0/2733125314 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5080cf050 0x7fb5080cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.782+0000 7fb50d38d640 1 -- 172.21.15.5:0/2733125314 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5080cf050 msgr2=0x7fb5080cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.782+0000 7fb50d38d640 1 --2- 172.21.15.5:0/2733125314 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5080cf050 0x7fb5080cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.782+0000 7fb50d38d640 1 -- 172.21.15.5:0/2733125314 >> v2:172.21.15.6:3300/0 conn(0x7fb50806c130 msgr2=0x7fb50806c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.782+0000 7fb50d38d640 1 --2- 172.21.15.5:0/2733125314 >> v2:172.21.15.6:3300/0 conn(0x7fb50806c130 0x7fb50806c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.782+0000 7fb50d38d640 1 -- 172.21.15.5:0/2733125314 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50815f140 con 0x7fb50813dfd0 2023-11-24T06:16:05.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.783+0000 7fb50d38d640 1 --2- 172.21.15.5:0/2733125314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb50813dfd0 0x7fb50815be70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb50806d520 tx=0x7fb50402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc9041e9938852c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:05.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.783+0000 7fb4fffff640 1 -- 172.21.15.5:0/2733125314 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504031870 con 0x7fb50813dfd0 2023-11-24T06:16:05.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.783+0000 7fb4fffff640 1 -- 172.21.15.5:0/2733125314 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5040319d0 con 0x7fb50813dfd0 2023-11-24T06:16:05.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.783+0000 7fb4fffff640 1 -- 172.21.15.5:0/2733125314 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504031d60 con 0x7fb50813dfd0 2023-11-24T06:16:05.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.784+0000 7fb50e38f640 1 -- 172.21.15.5:0/2733125314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb50813dfd0 msgr2=0x7fb50815be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:05.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.784+0000 7fb50e38f640 1 --2- 172.21.15.5:0/2733125314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb50813dfd0 0x7fb50815be70 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb50806d520 tx=0x7fb50402b910 comp rx=0 tx=0).stop 2023-11-24T06:16:05.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.784+0000 7fb50e38f640 1 -- 172.21.15.5:0/2733125314 shutdown_connections 2023-11-24T06:16:05.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.784+0000 7fb50e38f640 1 --2- 172.21.15.5:0/2733125314 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb5080cf050 0x7fb5080cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.784+0000 7fb50e38f640 1 --2- 172.21.15.5:0/2733125314 >> v2:172.21.15.6:3300/0 conn(0x7fb50806c130 0x7fb50806c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.784+0000 7fb50e38f640 1 --2- 172.21.15.5:0/2733125314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb50813dfd0 0x7fb50815be70 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.784+0000 7fb50e38f640 1 -- 172.21.15.5:0/2733125314 >> 172.21.15.5:0/2733125314 conn(0x7fb50806a860 msgr2=0x7fb50806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:05.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.784+0000 7fb50e38f640 1 -- 172.21.15.5:0/2733125314 shutdown_connections 2023-11-24T06:16:05.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.784+0000 7fb50e38f640 1 -- 172.21.15.5:0/2733125314 wait complete. 2023-11-24T06:16:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.786+0000 7fb50e38f640 1 Processor -- start 2023-11-24T06:16:05.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.786+0000 7fb50e38f640 1 -- start start 2023-11-24T06:16:05.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.787+0000 7fb50e38f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb50806c130 0x7fb5081f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:05.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.787+0000 7fb50e38f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5080cf050 0x7fb508155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:05.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.787+0000 7fb50e38f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb50813dfd0 0x7fb5080cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:05.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.787+0000 7fb50e38f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5081f4f20 con 0x7fb50806c130 2023-11-24T06:16:05.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.787+0000 7fb50e38f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080b2460 con 0x7fb5080cf050 2023-11-24T06:16:05.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.787+0000 7fb50e38f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5080b2740 con 0x7fb50813dfd0 2023-11-24T06:16:05.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.787+0000 7fb50d38d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb50806c130 0x7fb5081f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:05.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.787+0000 7fb50d38d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb50806c130 0x7fb5081f1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41580/0 (socket says 172.21.15.5:41580) 2023-11-24T06:16:05.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.787+0000 7fb50d38d640 1 -- 172.21.15.5:0/2089846721 learned_addr learned my addr 172.21.15.5:0/2089846721 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:05.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.788+0000 7fb50cb8c640 1 --2- 172.21.15.5:0/2089846721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5080cf050 0x7fb508155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:05.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.788+0000 7fb50d38d640 1 -- 172.21.15.5:0/2089846721 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb50813dfd0 msgr2=0x7fb5080cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:05.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.788+0000 7fb50d38d640 1 --2- 172.21.15.5:0/2089846721 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb50813dfd0 0x7fb5080cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.788+0000 7fb50d38d640 1 -- 172.21.15.5:0/2089846721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5080cf050 msgr2=0x7fb508155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:05.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.788+0000 7fb50d38d640 1 --2- 172.21.15.5:0/2089846721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5080cf050 0x7fb508155170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:05.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.788+0000 7fb50d38d640 1 -- 172.21.15.5:0/2089846721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5040094d0 con 0x7fb50806c130 2023-11-24T06:16:05.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.788+0000 7fb50d38d640 1 --2- 172.21.15.5:0/2089846721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb50806c130 0x7fb5081f1b50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb50402b950 tx=0x7fb5040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.788+0000 7fb4fdffb640 1 -- 172.21.15.5:0/2089846721 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50403ad30 con 0x7fb50806c130 2023-11-24T06:16:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.788+0000 7fb4fdffb640 1 -- 172.21.15.5:0/2089846721 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5040443f0 con 0x7fb50806c130 2023-11-24T06:16:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.788+0000 7fb4fdffb640 1 -- 172.21.15.5:0/2089846721 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5040447a0 con 0x7fb50806c130 2023-11-24T06:16:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.788+0000 7fb50e38f640 1 -- 172.21.15.5:0/2089846721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5080b2a20 con 0x7fb50806c130 2023-11-24T06:16:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.789+0000 7fb50e38f640 1 -- 172.21.15.5:0/2089846721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5081f2090 con 0x7fb50806c130 2023-11-24T06:16:05.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.790+0000 7fb50e38f640 1 -- 172.21.15.5:0/2089846721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb50804e840 con 0x7fb50806c130 2023-11-24T06:16:05.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.791+0000 7fb4fdffb640 1 -- 172.21.15.5:0/2089846721 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb50404d450 con 0x7fb50806c130 2023-11-24T06:16:05.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.792+0000 7fb4fdffb640 1 --2- 172.21.15.5:0/2089846721 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb4ec06ae70 0x7fb4ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:05.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.792+0000 7fb4fdffb640 1 -- 172.21.15.5:0/2089846721 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb50402d040 con 0x7fb50806c130 2023-11-24T06:16:05.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.792+0000 7fb50cb8c640 1 --2- 172.21.15.5:0/2089846721 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb4ec06ae70 0x7fb4ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:05.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.793+0000 7fb50cb8c640 1 --2- 172.21.15.5:0/2089846721 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb4ec06ae70 0x7fb4ec06d330 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb50815a940 tx=0x7fb4f8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:05.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.796+0000 7fb4fdffb640 1 -- 172.21.15.5:0/2089846721 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb504076bf0 con 0x7fb50806c130 2023-11-24T06:16:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.992+0000 7fb50e38f640 1 -- 172.21.15.5:0/2089846721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5080ce9a0 con 0x7fb50806c130 2023-11-24T06:16:05.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.995+0000 7fb4fdffb640 1 -- 172.21.15.5:0/2089846721 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16361 (secure 0 0 0) 0x7fb504035070 con 0x7fb50806c130 2023-11-24T06:16:05.998 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:05.999 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:52.779059+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:06.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.998+0000 7fb50e38f640 1 -- 172.21.15.5:0/2089846721 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb4ec06ae70 msgr2=0x7fb4ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:06.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.998+0000 7fb50e38f640 1 --2- 172.21.15.5:0/2089846721 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb4ec06ae70 0x7fb4ec06d330 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb50815a940 tx=0x7fb4f8008040 comp rx=0 tx=0).stop 2023-11-24T06:16:06.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.998+0000 7fb50e38f640 1 -- 172.21.15.5:0/2089846721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb50806c130 msgr2=0x7fb5081f1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:06.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.998+0000 7fb50e38f640 1 --2- 172.21.15.5:0/2089846721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb50806c130 0x7fb5081f1b50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb50402b950 tx=0x7fb5040046c0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.998+0000 7fb50e38f640 1 -- 172.21.15.5:0/2089846721 shutdown_connections 2023-11-24T06:16:06.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.998+0000 7fb50e38f640 1 --2- 172.21.15.5:0/2089846721 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb4ec06ae70 0x7fb4ec06d330 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.998+0000 7fb50e38f640 1 --2- 172.21.15.5:0/2089846721 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb50813dfd0 0x7fb5080cd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.998+0000 7fb50e38f640 1 --2- 172.21.15.5:0/2089846721 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb5080cf050 0x7fb508155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.998+0000 7fb50e38f640 1 --2- 172.21.15.5:0/2089846721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb50806c130 0x7fb5081f1b50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.998+0000 7fb50e38f640 1 -- 172.21.15.5:0/2089846721 >> 172.21.15.5:0/2089846721 conn(0x7fb50806a860 msgr2=0x7fb5081597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:06.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.998+0000 7fb50e38f640 1 -- 172.21.15.5:0/2089846721 shutdown_connections 2023-11-24T06:16:06.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:05.998+0000 7fb50e38f640 1 -- 172.21.15.5:0/2089846721 wait complete. 2023-11-24T06:16:06.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:06.022 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:06.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.244+0000 7f121de36640 1 Processor -- start 2023-11-24T06:16:06.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.244+0000 7f121de36640 1 -- start start 2023-11-24T06:16:06.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.246+0000 7f121de36640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f121809cb30 0x7f12180abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:06.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.246+0000 7f121de36640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f12180adb60 0x7f12180adf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:06.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.246+0000 7f121de36640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f121815b730 con 0x7f121815b350 2023-11-24T06:16:06.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.246+0000 7f121de36640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12180ac740 con 0x7f12180adb60 2023-11-24T06:16:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.246+0000 7f121de36640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12180ac460 con 0x7f121809cb30 2023-11-24T06:16:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.246+0000 7f12177fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f121815b350 0x7f12180b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37122/0 (socket says 172.21.15.5:37122) 2023-11-24T06:16:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.246+0000 7f12177fe640 1 -- 172.21.15.5:0/3737969874 learned_addr learned my addr 172.21.15.5:0/3737969874 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.246+0000 7f1217fff640 1 --2- 172.21.15.5:0/3737969874 >> v2:172.21.15.5:3300/0 conn(0x7f12180adb60 0x7f12180adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.247+0000 7f1216ffd640 1 --2- 172.21.15.5:0/3737969874 >> v2:172.21.15.6:3300/0 conn(0x7f121809cb30 0x7f12180abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:06.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.247+0000 7f12167fc640 1 -- 172.21.15.5:0/3737969874 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2105535930 0 0) 0x7f1208001110 con 0x7f121815b350 2023-11-24T06:16:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.247+0000 7f12167fc640 1 -- 172.21.15.5:0/3737969874 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1200003520 con 0x7f121815b350 2023-11-24T06:16:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.247+0000 7f1217fff640 1 -- 172.21.15.5:0/3737969874 >> v2:172.21.15.6:3300/0 conn(0x7f121809cb30 msgr2=0x7f12180abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.247+0000 7f1217fff640 1 --2- 172.21.15.5:0/3737969874 >> v2:172.21.15.6:3300/0 conn(0x7f121809cb30 0x7f12180abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.247+0000 7f1217fff640 1 -- 172.21.15.5:0/3737969874 >> v1:172.21.15.5:6789/0 conn(0x7f121815b350 legacy=0x7f12180b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:06.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.247+0000 7f1217fff640 1 -- 172.21.15.5:0/3737969874 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12180b2a50 con 0x7f12180adb60 2023-11-24T06:16:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.248+0000 7f1217fff640 1 --2- 172.21.15.5:0/3737969874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12180adb60 0x7f12180adf70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f12180d0060 tx=0x7f121002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe4a394cfcf55fe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:06.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.248+0000 7f12167fc640 1 -- 172.21.15.5:0/3737969874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121002cc80 con 0x7f12180adb60 2023-11-24T06:16:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.248+0000 7f12167fc640 1 -- 172.21.15.5:0/3737969874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121002cde0 con 0x7f12180adb60 2023-11-24T06:16:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.248+0000 7f12167fc640 1 -- 172.21.15.5:0/3737969874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1210034640 con 0x7f12180adb60 2023-11-24T06:16:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.249+0000 7f121de36640 1 -- 172.21.15.5:0/3737969874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12180adb60 msgr2=0x7f12180adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.249+0000 7f121de36640 1 --2- 172.21.15.5:0/3737969874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12180adb60 0x7f12180adf70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f12180d0060 tx=0x7f121002bed0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.249+0000 7f121de36640 1 -- 172.21.15.5:0/3737969874 shutdown_connections 2023-11-24T06:16:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.249+0000 7f121de36640 1 --2- 172.21.15.5:0/3737969874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12180adb60 0x7f12180adf70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.249+0000 7f121de36640 1 --2- 172.21.15.5:0/3737969874 >> v2:172.21.15.6:3300/0 conn(0x7f121809cb30 0x7f12180abe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.249+0000 7f121de36640 1 -- 172.21.15.5:0/3737969874 >> 172.21.15.5:0/3737969874 conn(0x7f12180b0ba0 msgr2=0x7f12180b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.249+0000 7f121de36640 1 -- 172.21.15.5:0/3737969874 shutdown_connections 2023-11-24T06:16:06.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.249+0000 7f121de36640 1 -- 172.21.15.5:0/3737969874 wait complete. 2023-11-24T06:16:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.251+0000 7f121de36640 1 Processor -- start 2023-11-24T06:16:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.251+0000 7f121de36640 1 -- start start 2023-11-24T06:16:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.252+0000 7f121de36640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f121809cb30 0x7f12181e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.252+0000 7f121de36640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12180adb60 0x7f1218141a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.252+0000 7f121de36640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f121815b350 0x7f12181585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.252+0000 7f121de36640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1218141fb0 con 0x7f12180adb60 2023-11-24T06:16:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.252+0000 7f121de36640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12181e5840 con 0x7f121809cb30 2023-11-24T06:16:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.252+0000 7f121de36640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12181e5b20 con 0x7f121815b350 2023-11-24T06:16:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.253+0000 7f1216ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12180adb60 0x7f1218141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.253+0000 7f1216ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12180adb60 0x7f1218141a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41596/0 (socket says 172.21.15.5:41596) 2023-11-24T06:16:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.253+0000 7f1216ffd640 1 -- 172.21.15.5:0/1428115119 learned_addr learned my addr 172.21.15.5:0/1428115119 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.253+0000 7f12177fe640 1 --2- 172.21.15.5:0/1428115119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f121809cb30 0x7f12181e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.253+0000 7f1216ffd640 1 -- 172.21.15.5:0/1428115119 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f121815b350 msgr2=0x7f12181585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.253+0000 7f1216ffd640 1 --2- 172.21.15.5:0/1428115119 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f121815b350 0x7f12181585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.253+0000 7f1216ffd640 1 -- 172.21.15.5:0/1428115119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f121809cb30 msgr2=0x7f12181e68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.253+0000 7f1216ffd640 1 --2- 172.21.15.5:0/1428115119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f121809cb30 0x7f12181e68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.253+0000 7f1216ffd640 1 -- 172.21.15.5:0/1428115119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1210009a70 con 0x7f12180adb60 2023-11-24T06:16:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.253+0000 7f12177fe640 1 --2- 172.21.15.5:0/1428115119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f121809cb30 0x7f12181e68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.253+0000 7f1216ffd640 1 --2- 172.21.15.5:0/1428115119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12180adb60 0x7f1218141a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f120c00d800 tx=0x7f120c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:06.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.254+0000 7f11fffff640 1 -- 172.21.15.5:0/1428115119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c00b6c0 con 0x7f12180adb60 2023-11-24T06:16:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.254+0000 7f11fffff640 1 -- 172.21.15.5:0/1428115119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120c00b820 con 0x7f12180adb60 2023-11-24T06:16:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.254+0000 7f121de36640 1 -- 172.21.15.5:0/1428115119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12181e5e00 con 0x7f12180adb60 2023-11-24T06:16:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.254+0000 7f11fffff640 1 -- 172.21.15.5:0/1428115119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c00bbb0 con 0x7f12180adb60 2023-11-24T06:16:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.254+0000 7f121de36640 1 -- 172.21.15.5:0/1428115119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12181e63c0 con 0x7f12180adb60 2023-11-24T06:16:06.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.255+0000 7f121de36640 1 -- 172.21.15.5:0/1428115119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12181e6550 con 0x7f12180adb60 2023-11-24T06:16:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.256+0000 7f11fffff640 1 -- 172.21.15.5:0/1428115119 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f120c00bd10 con 0x7f12180adb60 2023-11-24T06:16:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.257+0000 7f11fffff640 1 --2- 172.21.15.5:0/1428115119 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f120406ae70 0x7f120406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.257+0000 7f11fffff640 1 -- 172.21.15.5:0/1428115119 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f120c089c10 con 0x7f12180adb60 2023-11-24T06:16:06.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.257+0000 7f12177fe640 1 --2- 172.21.15.5:0/1428115119 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f120406ae70 0x7f120406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:06.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.258+0000 7f12177fe640 1 --2- 172.21.15.5:0/1428115119 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f120406ae70 0x7f120406d330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1208001270 tx=0x7f1208009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:06.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.261+0000 7f11fffff640 1 -- 172.21.15.5:0/1428115119 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f120c054c20 con 0x7f12180adb60 2023-11-24T06:16:06.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.450+0000 7f121de36640 1 -- 172.21.15.5:0/1428115119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f121809d200 con 0x7f12180adb60 2023-11-24T06:16:06.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.451+0000 7f11fffff640 1 -- 172.21.15.5:0/1428115119 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f120c058240 con 0x7f12180adb60 2023-11-24T06:16:06.454 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.453+0000 7f121de36640 1 -- 172.21.15.5:0/1428115119 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f120406ae70 msgr2=0x7f120406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.453+0000 7f121de36640 1 --2- 172.21.15.5:0/1428115119 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f120406ae70 0x7f120406d330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1208001270 tx=0x7f1208009040 comp rx=0 tx=0).stop 2023-11-24T06:16:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.453+0000 7f121de36640 1 -- 172.21.15.5:0/1428115119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12180adb60 msgr2=0x7f1218141a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.453+0000 7f121de36640 1 --2- 172.21.15.5:0/1428115119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12180adb60 0x7f1218141a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f120c00d800 tx=0x7f120c00dcc0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.453+0000 7f121de36640 1 -- 172.21.15.5:0/1428115119 shutdown_connections 2023-11-24T06:16:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.453+0000 7f121de36640 1 --2- 172.21.15.5:0/1428115119 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f120406ae70 0x7f120406d330 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.453+0000 7f121de36640 1 --2- 172.21.15.5:0/1428115119 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f121815b350 0x7f12181585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.453+0000 7f121de36640 1 --2- 172.21.15.5:0/1428115119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12180adb60 0x7f1218141a70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.453+0000 7f121de36640 1 --2- 172.21.15.5:0/1428115119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f121809cb30 0x7f12181e68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.454+0000 7f121de36640 1 -- 172.21.15.5:0/1428115119 >> 172.21.15.5:0/1428115119 conn(0x7f12180b0ba0 msgr2=0x7f121809bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.454+0000 7f121de36640 1 -- 172.21.15.5:0/1428115119 shutdown_connections 2023-11-24T06:16:06.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.454+0000 7f121de36640 1 -- 172.21.15.5:0/1428115119 wait complete. 2023-11-24T06:16:06.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.692+0000 7fea4af16640 1 Processor -- start 2023-11-24T06:16:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.692+0000 7fea4af16640 1 -- start start 2023-11-24T06:16:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.693+0000 7fea4af16640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea44162220 0x7fea440b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.694+0000 7fea4af16640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fea44162600 0x7fea44162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.694+0000 7fea4af16640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea44162fa0 con 0x7fea4406c3c0 2023-11-24T06:16:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.694+0000 7fea4af16640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea440706a0 con 0x7fea44162600 2023-11-24T06:16:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.694+0000 7fea4af16640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea44070c60 con 0x7fea44162220 2023-11-24T06:16:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.694+0000 7fea48cb2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea44162220 0x7fea440b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.694+0000 7fea43fff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fea44162600 0x7fea44162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.695+0000 7fea494b3640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7fea4406c3c0 0x7fea440710f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.5:36544/0 (socket says 172.21.15.5:36544) 2023-11-24T06:16:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.695+0000 7fea494b3640 1 -- 172.21.15.5:0/1481385447 learned_addr learned my addr 172.21.15.5:0/1481385447 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.695+0000 7fea437fe640 1 -- 172.21.15.5:0/1481385447 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259089700 0 0) 0x7fea44162fa0 con 0x7fea4406c3c0 2023-11-24T06:16:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.695+0000 7fea437fe640 1 -- 172.21.15.5:0/1481385447 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea2c003520 con 0x7fea4406c3c0 2023-11-24T06:16:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.695+0000 7fea48cb2640 1 -- 172.21.15.5:0/1481385447 >> v2:172.21.15.6:3300/0 conn(0x7fea44162600 msgr2=0x7fea44162a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.695+0000 7fea48cb2640 1 --2- 172.21.15.5:0/1481385447 >> v2:172.21.15.6:3300/0 conn(0x7fea44162600 0x7fea44162a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.695+0000 7fea48cb2640 1 -- 172.21.15.5:0/1481385447 >> v1:172.21.15.6:6789/0 conn(0x7fea4406c3c0 legacy=0x7fea440710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.695+0000 7fea48cb2640 1 -- 172.21.15.5:0/1481385447 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea440722b0 con 0x7fea44162220 2023-11-24T06:16:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.695+0000 7fea43fff640 1 --2- 172.21.15.5:0/1481385447 >> v2:172.21.15.6:3300/0 conn(0x7fea44162600 0x7fea44162a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:16:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.697+0000 7fea48cb2640 1 --2- 172.21.15.5:0/1481385447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea44162220 0x7fea440b1ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea34009d90 tx=0x7fea3402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a660c26e9b79774b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.697+0000 7fea437fe640 1 -- 172.21.15.5:0/1481385447 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3402cc50 con 0x7fea44162220 2023-11-24T06:16:06.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.698+0000 7fea437fe640 1 -- 172.21.15.5:0/1481385447 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3402cdb0 con 0x7fea44162220 2023-11-24T06:16:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.698+0000 7fea437fe640 1 -- 172.21.15.5:0/1481385447 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34034660 con 0x7fea44162220 2023-11-24T06:16:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.698+0000 7fea4af16640 1 -- 172.21.15.5:0/1481385447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea44162220 msgr2=0x7fea440b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.698+0000 7fea4af16640 1 --2- 172.21.15.5:0/1481385447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea44162220 0x7fea440b1ea0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea34009d90 tx=0x7fea3402bea0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.698+0000 7fea4af16640 1 -- 172.21.15.5:0/1481385447 shutdown_connections 2023-11-24T06:16:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.698+0000 7fea4af16640 1 --2- 172.21.15.5:0/1481385447 >> v2:172.21.15.6:3300/0 conn(0x7fea44162600 0x7fea44162a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.698+0000 7fea4af16640 1 --2- 172.21.15.5:0/1481385447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea44162220 0x7fea440b1ea0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.698+0000 7fea4af16640 1 -- 172.21.15.5:0/1481385447 >> 172.21.15.5:0/1481385447 conn(0x7fea440b0ba0 msgr2=0x7fea440b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.698+0000 7fea4af16640 1 -- 172.21.15.5:0/1481385447 shutdown_connections 2023-11-24T06:16:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.698+0000 7fea4af16640 1 -- 172.21.15.5:0/1481385447 wait complete. 2023-11-24T06:16:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.700+0000 7fea4af16640 1 Processor -- start 2023-11-24T06:16:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.700+0000 7fea4af16640 1 -- start start 2023-11-24T06:16:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.701+0000 7fea4af16640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea4406c3c0 0x7fea44157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.701+0000 7fea4af16640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea44162220 0x7fea440ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.701+0000 7fea4af16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44162600 0x7fea4409bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.701+0000 7fea4af16640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea440ce320 con 0x7fea44162600 2023-11-24T06:16:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.701+0000 7fea4af16640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea44157760 con 0x7fea44162220 2023-11-24T06:16:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.701+0000 7fea4af16640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea440cd710 con 0x7fea4406c3c0 2023-11-24T06:16:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.702+0000 7fea494b3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44162600 0x7fea4409bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.702+0000 7fea494b3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44162600 0x7fea4409bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41608/0 (socket says 172.21.15.5:41608) 2023-11-24T06:16:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.702+0000 7fea494b3640 1 -- 172.21.15.5:0/3524854934 learned_addr learned my addr 172.21.15.5:0/3524854934 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.702+0000 7fea43fff640 1 --2- 172.21.15.5:0/3524854934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea44162220 0x7fea440ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.702+0000 7fea48cb2640 1 --2- 172.21.15.5:0/3524854934 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea4406c3c0 0x7fea44157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.702+0000 7fea494b3640 1 -- 172.21.15.5:0/3524854934 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea4406c3c0 msgr2=0x7fea44157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.702+0000 7fea494b3640 1 --2- 172.21.15.5:0/3524854934 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea4406c3c0 0x7fea44157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.702+0000 7fea494b3640 1 -- 172.21.15.5:0/3524854934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea44162220 msgr2=0x7fea440ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.702+0000 7fea494b3640 1 --2- 172.21.15.5:0/3524854934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea44162220 0x7fea440ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.702+0000 7fea494b3640 1 -- 172.21.15.5:0/3524854934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea38009400 con 0x7fea44162600 2023-11-24T06:16:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.702+0000 7fea494b3640 1 --2- 172.21.15.5:0/3524854934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44162600 0x7fea4409bee0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c390 tx=0x7fea3c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.703+0000 7fea417fa640 1 -- 172.21.15.5:0/3524854934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c005ce0 con 0x7fea44162600 2023-11-24T06:16:06.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.703+0000 7fea417fa640 1 -- 172.21.15.5:0/3524854934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c005e40 con 0x7fea44162600 2023-11-24T06:16:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.703+0000 7fea417fa640 1 -- 172.21.15.5:0/3524854934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c007310 con 0x7fea44162600 2023-11-24T06:16:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.703+0000 7fea4af16640 1 -- 172.21.15.5:0/3524854934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea34009a70 con 0x7fea44162600 2023-11-24T06:16:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.703+0000 7fea4af16640 1 -- 172.21.15.5:0/3524854934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea440cdcd0 con 0x7fea44162600 2023-11-24T06:16:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.704+0000 7fea4af16640 1 -- 172.21.15.5:0/3524854934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea440ce490 con 0x7fea44162600 2023-11-24T06:16:06.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.706+0000 7fea417fa640 1 -- 172.21.15.5:0/3524854934 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea3c007470 con 0x7fea44162600 2023-11-24T06:16:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.707+0000 7fea417fa640 1 --2- 172.21.15.5:0/3524854934 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fea3006aec0 0x7fea3006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.707+0000 7fea417fa640 1 -- 172.21.15.5:0/3524854934 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea3c088c50 con 0x7fea44162600 2023-11-24T06:16:06.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.707+0000 7fea48cb2640 1 --2- 172.21.15.5:0/3524854934 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fea3006aec0 0x7fea3006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:06.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.708+0000 7fea48cb2640 1 --2- 172.21.15.5:0/3524854934 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fea3006aec0 0x7fea3006d380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fea34036000 tx=0x7fea34030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:06.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.712+0000 7fea417fa640 1 -- 172.21.15.5:0/3524854934 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea3c053ce0 con 0x7fea44162600 2023-11-24T06:16:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.906+0000 7fea4af16640 1 -- 172.21.15.5:0/3524854934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea4409d200 con 0x7fea44162600 2023-11-24T06:16:06.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.907+0000 7fea417fa640 1 -- 172.21.15.5:0/3524854934 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fea3c017020 con 0x7fea44162600 2023-11-24T06:16:06.910 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.909+0000 7fea4af16640 1 -- 172.21.15.5:0/3524854934 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fea3006aec0 msgr2=0x7fea3006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.909+0000 7fea4af16640 1 --2- 172.21.15.5:0/3524854934 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fea3006aec0 0x7fea3006d380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fea34036000 tx=0x7fea34030000 comp rx=0 tx=0).stop 2023-11-24T06:16:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.909+0000 7fea4af16640 1 -- 172.21.15.5:0/3524854934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44162600 msgr2=0x7fea4409bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.909+0000 7fea4af16640 1 --2- 172.21.15.5:0/3524854934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44162600 0x7fea4409bee0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c390 tx=0x7fea3c00c850 comp rx=0 tx=0).stop 2023-11-24T06:16:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.910+0000 7fea4af16640 1 -- 172.21.15.5:0/3524854934 shutdown_connections 2023-11-24T06:16:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.910+0000 7fea4af16640 1 --2- 172.21.15.5:0/3524854934 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fea3006aec0 0x7fea3006d380 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.910+0000 7fea4af16640 1 --2- 172.21.15.5:0/3524854934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea44162600 0x7fea4409bee0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.910+0000 7fea4af16640 1 --2- 172.21.15.5:0/3524854934 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fea44162220 0x7fea440ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.910+0000 7fea4af16640 1 --2- 172.21.15.5:0/3524854934 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fea4406c3c0 0x7fea44157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.910+0000 7fea4af16640 1 -- 172.21.15.5:0/3524854934 >> 172.21.15.5:0/3524854934 conn(0x7fea440b0ba0 msgr2=0x7fea44160f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.910+0000 7fea4af16640 1 -- 172.21.15.5:0/3524854934 shutdown_connections 2023-11-24T06:16:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:06.910+0000 7fea4af16640 1 -- 172.21.15.5:0/3524854934 wait complete. 2023-11-24T06:16:06.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:07.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:16:07.125 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:16:07.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.142+0000 7fe3ce4f3640 1 Processor -- start 2023-11-24T06:16:07.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.143+0000 7fe3ce4f3640 1 -- start start 2023-11-24T06:16:07.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.144+0000 7fe3ce4f3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3c8160bd0 0x7fe3c80b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:07.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.144+0000 7fe3ce4f3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3c80b3210 0x7fe3c8167290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:07.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.144+0000 7fe3ce4f3640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c80b23e0 con 0x7fe3c80b48d0 2023-11-24T06:16:07.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.144+0000 7fe3ce4f3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c80cdc70 con 0x7fe3c80b3210 2023-11-24T06:16:07.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.144+0000 7fe3ce4f3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80b2870 con 0x7fe3c8160bd0 2023-11-24T06:16:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.144+0000 7fe3cca90640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3c80b3210 0x7fe3c8167290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.144+0000 7fe3c7fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3c8160bd0 0x7fe3c80b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.144+0000 7fe3cca90640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3c80b3210 0x7fe3c8167290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41624/0 (socket says 172.21.15.5:41624) 2023-11-24T06:16:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.144+0000 7fe3cca90640 1 -- 172.21.15.5:0/2222742348 learned_addr learned my addr 172.21.15.5:0/2222742348 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.145+0000 7fe3cca90640 1 -- 172.21.15.5:0/2222742348 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3c8160bd0 msgr2=0x7fe3c80b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.145+0000 7fe3cca90640 1 --2- 172.21.15.5:0/2222742348 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3c8160bd0 0x7fe3c80b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.145+0000 7fe3cca90640 1 -- 172.21.15.5:0/2222742348 >> v1:172.21.15.6:6789/0 conn(0x7fe3c80b48d0 legacy=0x7fe3c80b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:07.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.145+0000 7fe3cca90640 1 -- 172.21.15.5:0/2222742348 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c80d41b0 con 0x7fe3c80b3210 2023-11-24T06:16:07.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.146+0000 7fe3cca90640 1 --2- 172.21.15.5:0/2222742348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c80b3210 0x7fe3c8167290 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009870 tx=0x7fe3c002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c96c5554acde71 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:07.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.147+0000 7fe3c77fe640 1 -- 172.21.15.5:0/2222742348 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0031640 con 0x7fe3c80b3210 2023-11-24T06:16:07.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.147+0000 7fe3c77fe640 1 -- 172.21.15.5:0/2222742348 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c00317a0 con 0x7fe3c80b3210 2023-11-24T06:16:07.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.147+0000 7fe3c77fe640 1 -- 172.21.15.5:0/2222742348 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0031b30 con 0x7fe3c80b3210 2023-11-24T06:16:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.148+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/2222742348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c80b3210 msgr2=0x7fe3c8167290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.148+0000 7fe3ce4f3640 1 --2- 172.21.15.5:0/2222742348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c80b3210 0x7fe3c8167290 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009870 tx=0x7fe3c002b8f0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.148+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/2222742348 shutdown_connections 2023-11-24T06:16:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.148+0000 7fe3ce4f3640 1 --2- 172.21.15.5:0/2222742348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c80b3210 0x7fe3c8167290 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.148+0000 7fe3ce4f3640 1 --2- 172.21.15.5:0/2222742348 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3c8160bd0 0x7fe3c80b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.148+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/2222742348 >> 172.21.15.5:0/2222742348 conn(0x7fe3c809c5a0 msgr2=0x7fe3c80d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:07.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.148+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/2222742348 shutdown_connections 2023-11-24T06:16:07.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.148+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/2222742348 wait complete. 2023-11-24T06:16:07.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.150+0000 7fe3ce4f3640 1 Processor -- start 2023-11-24T06:16:07.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.150+0000 7fe3ce4f3640 1 -- start start 2023-11-24T06:16:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.151+0000 7fe3ce4f3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3c80b3210 0x7fe3c80c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.151+0000 7fe3ce4f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c80b48d0 0x7fe3c8149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.151+0000 7fe3ce4f3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3c8160bd0 0x7fe3c81f7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.151+0000 7fe3ce4f3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80cc0b0 con 0x7fe3c80b48d0 2023-11-24T06:16:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.151+0000 7fe3ce4f3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80cb710 con 0x7fe3c8160bd0 2023-11-24T06:16:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.151+0000 7fe3ce4f3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80cb9f0 con 0x7fe3c80b3210 2023-11-24T06:16:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.151+0000 7fe3bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c80b48d0 0x7fe3c8149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.151+0000 7fe3bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c80b48d0 0x7fe3c8149080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41640/0 (socket says 172.21.15.5:41640) 2023-11-24T06:16:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.151+0000 7fe3bf7fe640 1 -- 172.21.15.5:0/3137541145 learned_addr learned my addr 172.21.15.5:0/3137541145 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.151+0000 7fe3c7fff640 1 --2- 172.21.15.5:0/3137541145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3c80b3210 0x7fe3c80c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.152+0000 7fe3cca90640 1 --2- 172.21.15.5:0/3137541145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3c8160bd0 0x7fe3c81f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:07.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.152+0000 7fe3bf7fe640 1 -- 172.21.15.5:0/3137541145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3c80b3210 msgr2=0x7fe3c80c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.152+0000 7fe3bf7fe640 1 --2- 172.21.15.5:0/3137541145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3c80b3210 0x7fe3c80c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.152+0000 7fe3bf7fe640 1 -- 172.21.15.5:0/3137541145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3c8160bd0 msgr2=0x7fe3c81f7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.152+0000 7fe3bf7fe640 1 --2- 172.21.15.5:0/3137541145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3c8160bd0 0x7fe3c81f7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.152+0000 7fe3bf7fe640 1 -- 172.21.15.5:0/3137541145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c00094d0 con 0x7fe3c80b48d0 2023-11-24T06:16:07.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.152+0000 7fe3bf7fe640 1 --2- 172.21.15.5:0/3137541145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c80b48d0 0x7fe3c8149080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000c400 tx=0x7fe3b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.153+0000 7fe3c57fa640 1 -- 172.21.15.5:0/3137541145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00052d0 con 0x7fe3c80b48d0 2023-11-24T06:16:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.153+0000 7fe3c57fa640 1 -- 172.21.15.5:0/3137541145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b0005430 con 0x7fe3c80b48d0 2023-11-24T06:16:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.153+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/3137541145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c80cbcd0 con 0x7fe3c80b48d0 2023-11-24T06:16:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.153+0000 7fe3c57fa640 1 -- 172.21.15.5:0/3137541145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b0003830 con 0x7fe3c80b48d0 2023-11-24T06:16:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.153+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/3137541145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c81495c0 con 0x7fe3c80b48d0 2023-11-24T06:16:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.154+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/3137541145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c80cc1f0 con 0x7fe3c80b48d0 2023-11-24T06:16:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.156+0000 7fe3c57fa640 1 -- 172.21.15.5:0/3137541145 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe3b0003990 con 0x7fe3c80b48d0 2023-11-24T06:16:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.157+0000 7fe3c57fa640 1 --2- 172.21.15.5:0/3137541145 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe3a806ae90 0x7fe3a806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.157+0000 7fe3c57fa640 1 -- 172.21.15.5:0/3137541145 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3b0088b50 con 0x7fe3c80b48d0 2023-11-24T06:16:07.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.157+0000 7fe3c7fff640 1 --2- 172.21.15.5:0/3137541145 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe3a806ae90 0x7fe3a806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:07.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.158+0000 7fe3c7fff640 1 --2- 172.21.15.5:0/3137541145 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe3a806ae90 0x7fe3a806d350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8011340 tx=0x7fe3b800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:07.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.163+0000 7fe3c57fa640 1 -- 172.21.15.5:0/3137541145 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3c80cc1f0 con 0x7fe3c80b48d0 2023-11-24T06:16:07.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.358+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/3137541145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3c80cc1f0 con 0x7fe3c80b48d0 2023-11-24T06:16:07.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.359+0000 7fe3c57fa640 1 -- 172.21.15.5:0/3137541145 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe3c80cc1f0 con 0x7fe3c80b48d0 2023-11-24T06:16:07.362 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.362+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/3137541145 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe3a806ae90 msgr2=0x7fe3a806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.362+0000 7fe3ce4f3640 1 --2- 172.21.15.5:0/3137541145 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe3a806ae90 0x7fe3a806d350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8011340 tx=0x7fe3b800b040 comp rx=0 tx=0).stop 2023-11-24T06:16:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.362+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/3137541145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c80b48d0 msgr2=0x7fe3c8149080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.362+0000 7fe3ce4f3640 1 --2- 172.21.15.5:0/3137541145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c80b48d0 0x7fe3c8149080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe3b000c400 tx=0x7fe3b000c8c0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.362+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/3137541145 shutdown_connections 2023-11-24T06:16:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.362+0000 7fe3ce4f3640 1 --2- 172.21.15.5:0/3137541145 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe3a806ae90 0x7fe3a806d350 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.362+0000 7fe3ce4f3640 1 --2- 172.21.15.5:0/3137541145 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3c8160bd0 0x7fe3c81f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.362+0000 7fe3ce4f3640 1 --2- 172.21.15.5:0/3137541145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c80b48d0 0x7fe3c8149080 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.362+0000 7fe3ce4f3640 1 --2- 172.21.15.5:0/3137541145 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3c80b3210 0x7fe3c80c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.362+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/3137541145 >> 172.21.15.5:0/3137541145 conn(0x7fe3c809c5a0 msgr2=0x7fe3c815f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:07.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.362+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/3137541145 shutdown_connections 2023-11-24T06:16:07.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.362+0000 7fe3ce4f3640 1 -- 172.21.15.5:0/3137541145 wait complete. 2023-11-24T06:16:07.384 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:16:07.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.605+0000 7f270107e640 1 Processor -- start 2023-11-24T06:16:07.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.605+0000 7f270107e640 1 -- start start 2023-11-24T06:16:07.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.606+0000 7f270107e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f26fc13dfd0 0x7f26fc15be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:07.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.606+0000 7f270107e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26fc0ce7c0 con 0x7f26fc0cf500 2023-11-24T06:16:07.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.606+0000 7f270107e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26fc140900 con 0x7f26fc0cf050 2023-11-24T06:16:07.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.606+0000 7f270107e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26fc06cf50 con 0x7f26fc13dfd0 2023-11-24T06:16:07.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.607+0000 7f270087d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f26fc0cf500 0x7f26fc06c840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37124/0 (socket says 172.21.15.5:37124) 2023-11-24T06:16:07.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.607+0000 7f270087d640 1 -- 172.21.15.5:0/664775157 learned_addr learned my addr 172.21.15.5:0/664775157 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:07.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.607+0000 7f26fbfff640 1 --2- 172.21.15.5:0/664775157 >> v2:172.21.15.6:3300/0 conn(0x7f26fc13dfd0 0x7f26fc15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:07.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.607+0000 7f26faffd640 1 -- 172.21.15.5:0/664775157 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1992085787 0 0) 0x7f26fc0ce7c0 con 0x7f26fc0cf500 2023-11-24T06:16:07.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.607+0000 7f26faffd640 1 -- 172.21.15.5:0/664775157 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e4003520 con 0x7f26fc0cf500 2023-11-24T06:16:07.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.608+0000 7f26faffd640 1 -- 172.21.15.5:0/664775157 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2656729865 0 0) 0x7f26e4003520 con 0x7f26fc0cf500 2023-11-24T06:16:07.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.608+0000 7f26faffd640 1 -- 172.21.15.5:0/664775157 >> v2:172.21.15.6:3300/0 conn(0x7f26fc13dfd0 msgr2=0x7f26fc15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:07.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.608+0000 7f26faffd640 1 --2- 172.21.15.5:0/664775157 >> v2:172.21.15.6:3300/0 conn(0x7f26fc13dfd0 0x7f26fc15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.608+0000 7f26faffd640 1 -- 172.21.15.5:0/664775157 >> v1:172.21.15.6:6789/0 conn(0x7f26fc0cf050 legacy=0x7f26fc06c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:07.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.608+0000 7f26faffd640 1 -- 172.21.15.5:0/664775157 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc15fa70 con 0x7f26fc0cf500 2023-11-24T06:16:07.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.608+0000 7f26faffd640 1 -- 172.21.15.5:0/664775157 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f26f40019f0 con 0x7f26fc0cf500 2023-11-24T06:16:07.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.608+0000 7f26faffd640 1 -- 172.21.15.5:0/664775157 >> v1:172.21.15.5:6789/0 conn(0x7f26fc0cf500 legacy=0x7f26fc06c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:07.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.608+0000 7f26faffd640 1 --2- 172.21.15.5:0/664775157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e4003a30 0x7f26e4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.608+0000 7f26faffd640 1 --2- 172.21.15.5:0/664775157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f26e40047c0 0x7f26e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.608+0000 7f26faffd640 1 --2- 172.21.15.5:0/664775157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e40051d0 0x7f26e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.608+0000 7f26faffd640 1 -- 172.21.15.5:0/664775157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc06cf50 con 0x7f26e4003a30 2023-11-24T06:16:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.608+0000 7f26faffd640 1 -- 172.21.15.5:0/664775157 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e400c100 con 0x7f26e40051d0 2023-11-24T06:16:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.608+0000 7f26faffd640 1 -- 172.21.15.5:0/664775157 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e400c2c0 con 0x7f26e40047c0 2023-11-24T06:16:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.608+0000 7f26fbfff640 1 --2- 172.21.15.5:0/664775157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e4003a30 0x7f26e4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.609+0000 7f26fbfff640 1 -- 172.21.15.5:0/664775157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f26e40047c0 msgr2=0x7f26e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.609+0000 7f26fbfff640 1 --2- 172.21.15.5:0/664775157 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f26e40047c0 0x7f26e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.609+0000 7f26fbfff640 1 -- 172.21.15.5:0/664775157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e40051d0 msgr2=0x7f26e400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.609+0000 7f26fbfff640 1 --2- 172.21.15.5:0/664775157 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e40051d0 0x7f26e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.609+0000 7f26fbfff640 1 -- 172.21.15.5:0/664775157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e4003520 con 0x7f26e4003a30 2023-11-24T06:16:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.609+0000 7f270087d640 1 -- 172.21.15.5:0/664775157 reap_dead start 2023-11-24T06:16:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.610+0000 7f26fbfff640 1 --2- 172.21.15.5:0/664775157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e4003a30 0x7f26e4003e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009ac0 tx=0x7f26e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bf51151bce98c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.610+0000 7f26faffd640 1 -- 172.21.15.5:0/664775157 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec02c710 con 0x7f26e4003a30 2023-11-24T06:16:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.610+0000 7f26faffd640 1 -- 172.21.15.5:0/664775157 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec02c870 con 0x7f26e4003a30 2023-11-24T06:16:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.610+0000 7f26faffd640 1 -- 172.21.15.5:0/664775157 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec02cc00 con 0x7f26e4003a30 2023-11-24T06:16:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.611+0000 7f270107e640 1 -- 172.21.15.5:0/664775157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e4003a30 msgr2=0x7f26e4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.611+0000 7f270107e640 1 --2- 172.21.15.5:0/664775157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e4003a30 0x7f26e4003e50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009ac0 tx=0x7f26e40057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.611+0000 7f270107e640 1 -- 172.21.15.5:0/664775157 shutdown_connections 2023-11-24T06:16:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.611+0000 7f270107e640 1 --2- 172.21.15.5:0/664775157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e4003a30 0x7f26e4003e50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.611+0000 7f270107e640 1 -- 172.21.15.5:0/664775157 >> 172.21.15.5:0/664775157 conn(0x7f26fc06a860 msgr2=0x7f26fc06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.611+0000 7f270107e640 1 -- 172.21.15.5:0/664775157 shutdown_connections 2023-11-24T06:16:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.611+0000 7f270107e640 1 -- 172.21.15.5:0/664775157 wait complete. 2023-11-24T06:16:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.613+0000 7f270107e640 1 Processor -- start 2023-11-24T06:16:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.613+0000 7f270107e640 1 -- start start 2023-11-24T06:16:07.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f270107e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4003a30 0x7f26fc0b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:07.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f270107e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f26fc0b2f80 0x7f26fc1f08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f270107e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26fc159eb0 0x7f26fc15a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f270107e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc137250 con 0x7f26fc159eb0 2023-11-24T06:16:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f270107e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc0b3400 con 0x7f26e4003a30 2023-11-24T06:16:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f270107e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26fc15a850 con 0x7f26fc0b2f80 2023-11-24T06:16:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f270087d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26fc159eb0 0x7f26fc15a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f270087d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26fc159eb0 0x7f26fc15a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41652/0 (socket says 172.21.15.5:41652) 2023-11-24T06:16:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f270087d640 1 -- 172.21.15.5:0/4014440682 learned_addr learned my addr 172.21.15.5:0/4014440682 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f26fbfff640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4003a30 0x7f26fc0b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f26fb7fe640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f26fc0b2f80 0x7f26fc1f08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f270087d640 1 -- 172.21.15.5:0/4014440682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f26fc0b2f80 msgr2=0x7f26fc1f08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f270087d640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f26fc0b2f80 0x7f26fc1f08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f270087d640 1 -- 172.21.15.5:0/4014440682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4003a30 msgr2=0x7f26fc0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:07.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f270087d640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4003a30 0x7f26fc0b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.615+0000 7f270087d640 1 -- 172.21.15.5:0/4014440682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc0b3620 con 0x7f26fc159eb0 2023-11-24T06:16:07.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.616+0000 7f26fbfff640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4003a30 0x7f26fc0b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:16:07.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.616+0000 7f270087d640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26fc159eb0 0x7f26fc15a310 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0cfd30 tx=0x7f26f400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.616+0000 7f26f8ff9640 1 -- 172.21.15.5:0/4014440682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f4004d70 con 0x7f26fc159eb0 2023-11-24T06:16:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.616+0000 7f26f8ff9640 1 -- 172.21.15.5:0/4014440682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f400a5b0 con 0x7f26fc159eb0 2023-11-24T06:16:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.616+0000 7f270107e640 1 -- 172.21.15.5:0/4014440682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26fc1368b0 con 0x7f26fc159eb0 2023-11-24T06:16:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.616+0000 7f26f8ff9640 1 -- 172.21.15.5:0/4014440682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f400a960 con 0x7f26fc159eb0 2023-11-24T06:16:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.616+0000 7f270107e640 1 -- 172.21.15.5:0/4014440682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26fc136e70 con 0x7f26fc159eb0 2023-11-24T06:16:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.617+0000 7f270107e640 1 -- 172.21.15.5:0/4014440682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26fc04e840 con 0x7f26fc159eb0 2023-11-24T06:16:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.618+0000 7f26f8ff9640 1 -- 172.21.15.5:0/4014440682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f26f400aac0 con 0x7f26fc159eb0 2023-11-24T06:16:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.619+0000 7f26f8ff9640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f26e806ae40 0x7f26e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.619+0000 7f26f8ff9640 1 -- 172.21.15.5:0/4014440682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26f4089170 con 0x7f26fc159eb0 2023-11-24T06:16:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.620+0000 7f26fbfff640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f26e806ae40 0x7f26e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.620+0000 7f26fbfff640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f26e806ae40 0x7f26e806d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f26ec036000 tx=0x7f26ec030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:07.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.624+0000 7f26f8ff9640 1 -- 172.21.15.5:0/4014440682 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26f40542b0 con 0x7f26fc159eb0 2023-11-24T06:16:07.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.866+0000 7f270107e640 1 -- 172.21.15.5:0/4014440682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26fc0ce9a0 con 0x7f26fc159eb0 2023-11-24T06:16:07.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.867+0000 7f26f8ff9640 1 -- 172.21.15.5:0/4014440682 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4895 (secure 0 0 0) 0x7f26f40578d0 con 0x7f26fc159eb0 2023-11-24T06:16:07.870 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:07.871 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4009460946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4009460946},{"type":"v1","addr":"172.21.15.5:6837","nonce":4009460946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2496926427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2496926427},{"type":"v1","addr":"172.21.15.5:6836","nonce":2496926427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3829754769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3829754769},{"type":"v1","addr":"172.21.15.6:6835","nonce":3829754769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:15:51.611541+0000","modified":"2023-11-24T06:15:54.664234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6833/2760317954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2760317954},{"type":"v1","addr":"172.21.15.6:6833","nonce":2760317954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:16:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2023-11-24T06:16:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.870+0000 7f270107e640 1 -- 172.21.15.5:0/4014440682 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f26e806ae40 msgr2=0x7f26e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.870+0000 7f270107e640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f26e806ae40 0x7f26e806d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f26ec036000 tx=0x7f26ec030000 comp rx=0 tx=0).stop 2023-11-24T06:16:07.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.870+0000 7f270107e640 1 -- 172.21.15.5:0/4014440682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26fc159eb0 msgr2=0x7f26fc15a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:07.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.870+0000 7f270107e640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26fc159eb0 0x7f26fc15a310 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f26fc0cfd30 tx=0x7f26f400cb90 comp rx=0 tx=0).stop 2023-11-24T06:16:07.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.870+0000 7f270107e640 1 -- 172.21.15.5:0/4014440682 shutdown_connections 2023-11-24T06:16:07.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.870+0000 7f270107e640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f26e806ae40 0x7f26e806d300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.870+0000 7f270107e640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f26fc0b2f80 0x7f26fc1f08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.870+0000 7f270107e640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26fc159eb0 0x7f26fc15a310 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.870+0000 7f270107e640 1 --2- 172.21.15.5:0/4014440682 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f26e4003a30 0x7f26fc0b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.870+0000 7f270107e640 1 -- 172.21.15.5:0/4014440682 >> 172.21.15.5:0/4014440682 conn(0x7f26fc06a860 msgr2=0x7f26fc06c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.870+0000 7f270107e640 1 -- 172.21.15.5:0/4014440682 shutdown_connections 2023-11-24T06:16:07.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:07.870+0000 7f270107e640 1 -- 172.21.15.5:0/4014440682 wait complete. 2023-11-24T06:16:07.894 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.122+0000 7f79548ac640 1 Processor -- start 2023-11-24T06:16:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.122+0000 7f79548ac640 1 -- start start 2023-11-24T06:16:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.124+0000 7f79548ac640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7950143950 0x7f7950164e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.124+0000 7f79548ac640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7950071700 con 0x7f79500cf760 2023-11-24T06:16:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.124+0000 7f79548ac640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f795015b410 con 0x7f7950143570 2023-11-24T06:16:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.124+0000 7f79548ac640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f795015b130 con 0x7f7950143950 2023-11-24T06:16:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.124+0000 7f7947fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f79500cf760 0x7f795015a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:37140/0 (socket says 172.21.15.5:37140) 2023-11-24T06:16:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.124+0000 7f7947fff640 1 -- 172.21.15.5:0/2510568454 learned_addr learned my addr 172.21.15.5:0/2510568454 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.124+0000 7f794f59e640 1 --2- 172.21.15.5:0/2510568454 >> v2:172.21.15.5:3300/0 conn(0x7f7950143950 0x7f7950164e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.125+0000 7f794f59e640 1 -- 172.21.15.5:0/2510568454 >> v1:172.21.15.6:6789/0 conn(0x7f7950143570 legacy=0x7f79500cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.125+0000 7f794f59e640 1 -- 172.21.15.5:0/2510568454 >> v1:172.21.15.5:6789/0 conn(0x7f79500cf760 legacy=0x7f795015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.125+0000 7f794f59e640 1 -- 172.21.15.5:0/2510568454 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795015dbd0 con 0x7f7950143950 2023-11-24T06:16:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.126+0000 7f794f59e640 1 --2- 172.21.15.5:0/2510568454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950143950 0x7f7950164e30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7948009d90 tx=0x7f794802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10aef97ddc3feb30 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.126+0000 7f794e59c640 1 -- 172.21.15.5:0/2510568454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794802cbd0 con 0x7f7950143950 2023-11-24T06:16:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.126+0000 7f794e59c640 1 -- 172.21.15.5:0/2510568454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794802cd30 con 0x7f7950143950 2023-11-24T06:16:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.126+0000 7f794e59c640 1 -- 172.21.15.5:0/2510568454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794802cea0 con 0x7f7950143950 2023-11-24T06:16:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.127+0000 7f79548ac640 1 -- 172.21.15.5:0/2510568454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950143950 msgr2=0x7f7950164e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.127+0000 7f79548ac640 1 --2- 172.21.15.5:0/2510568454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950143950 0x7f7950164e30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7948009d90 tx=0x7f794802be20 comp rx=0 tx=0).stop 2023-11-24T06:16:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.127+0000 7f79548ac640 1 -- 172.21.15.5:0/2510568454 shutdown_connections 2023-11-24T06:16:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.127+0000 7f79548ac640 1 --2- 172.21.15.5:0/2510568454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950143950 0x7f7950164e30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.127+0000 7f79548ac640 1 -- 172.21.15.5:0/2510568454 >> 172.21.15.5:0/2510568454 conn(0x7f795006a860 msgr2=0x7f795006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.127+0000 7f79548ac640 1 -- 172.21.15.5:0/2510568454 shutdown_connections 2023-11-24T06:16:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.127+0000 7f79548ac640 1 -- 172.21.15.5:0/2510568454 wait complete. 2023-11-24T06:16:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.129+0000 7f79548ac640 1 Processor -- start 2023-11-24T06:16:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.129+0000 7f79548ac640 1 -- start start 2023-11-24T06:16:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.130+0000 7f79548ac640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79500cf760 0x7f7950159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.130+0000 7f79548ac640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7950143570 0x7f79500cdd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.130+0000 7f79548ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950143950 0x7f795006c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.130+0000 7f79548ac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79500c8ae0 con 0x7f7950143950 2023-11-24T06:16:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.130+0000 7f79548ac640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79501e5f00 con 0x7f7950143570 2023-11-24T06:16:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.130+0000 7f79548ac640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7950159d50 con 0x7f79500cf760 2023-11-24T06:16:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.130+0000 7f794f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950143950 0x7f795006c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.130+0000 7f794f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950143950 0x7f795006c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41660/0 (socket says 172.21.15.5:41660) 2023-11-24T06:16:08.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.130+0000 7f794f59e640 1 -- 172.21.15.5:0/946292306 learned_addr learned my addr 172.21.15.5:0/946292306 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.131+0000 7f7947fff640 1 --2- 172.21.15.5:0/946292306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7950143570 0x7f79500cdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.131+0000 7f794ed9d640 1 --2- 172.21.15.5:0/946292306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79500cf760 0x7f7950159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.131+0000 7f794f59e640 1 -- 172.21.15.5:0/946292306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79500cf760 msgr2=0x7f7950159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.131+0000 7f794f59e640 1 --2- 172.21.15.5:0/946292306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79500cf760 0x7f7950159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.131+0000 7f794f59e640 1 -- 172.21.15.5:0/946292306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7950143570 msgr2=0x7f79500cdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.131+0000 7f794f59e640 1 --2- 172.21.15.5:0/946292306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7950143570 0x7f79500cdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.131+0000 7f794f59e640 1 -- 172.21.15.5:0/946292306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7948009a70 con 0x7f7950143950 2023-11-24T06:16:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.131+0000 7f794f59e640 1 --2- 172.21.15.5:0/946292306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950143950 0x7f795006c130 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7948009d90 tx=0x7f7948004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.132+0000 7f79477fe640 1 -- 172.21.15.5:0/946292306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794802cea0 con 0x7f7950143950 2023-11-24T06:16:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.132+0000 7f79477fe640 1 -- 172.21.15.5:0/946292306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794803e630 con 0x7f7950143950 2023-11-24T06:16:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.132+0000 7f79548ac640 1 -- 172.21.15.5:0/946292306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79500c7f60 con 0x7f7950143950 2023-11-24T06:16:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.132+0000 7f79477fe640 1 -- 172.21.15.5:0/946292306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794803e9e0 con 0x7f7950143950 2023-11-24T06:16:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.132+0000 7f79548ac640 1 -- 172.21.15.5:0/946292306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79500c8520 con 0x7f7950143950 2023-11-24T06:16:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.133+0000 7f79548ac640 1 -- 172.21.15.5:0/946292306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79501e6070 con 0x7f7950143950 2023-11-24T06:16:08.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.134+0000 7f79477fe640 1 -- 172.21.15.5:0/946292306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f794803b050 con 0x7f7950143950 2023-11-24T06:16:08.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.135+0000 7f79477fe640 1 --2- 172.21.15.5:0/946292306 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f793006ae40 0x7f793006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:08.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.136+0000 7f79477fe640 1 -- 172.21.15.5:0/946292306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79480ab820 con 0x7f7950143950 2023-11-24T06:16:08.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.136+0000 7f794ed9d640 1 --2- 172.21.15.5:0/946292306 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f793006ae40 0x7f793006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:08.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.136+0000 7f794ed9d640 1 --2- 172.21.15.5:0/946292306 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f793006ae40 0x7f793006d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7940004ea0 tx=0x7f7940009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:08.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.140+0000 7f79477fe640 1 -- 172.21.15.5:0/946292306 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f79501e6070 con 0x7f7950143950 2023-11-24T06:16:08.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.334+0000 7f79548ac640 1 -- 172.21.15.5:0/946292306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79501e6070 con 0x7f7950143950 2023-11-24T06:16:08.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.337+0000 7f79477fe640 1 -- 172.21.15.5:0/946292306 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16361 (secure 0 0 0) 0x7f79501e6070 con 0x7f7950143950 2023-11-24T06:16:08.448 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:08.448 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:52.779059+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.339+0000 7f79548ac640 1 -- 172.21.15.5:0/946292306 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f793006ae40 msgr2=0x7f793006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.339+0000 7f79548ac640 1 --2- 172.21.15.5:0/946292306 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f793006ae40 0x7f793006d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7940004ea0 tx=0x7f7940009040 comp rx=0 tx=0).stop 2023-11-24T06:16:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.339+0000 7f79548ac640 1 -- 172.21.15.5:0/946292306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950143950 msgr2=0x7f795006c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.339+0000 7f79548ac640 1 --2- 172.21.15.5:0/946292306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950143950 0x7f795006c130 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7948009d90 tx=0x7f7948004a40 comp rx=0 tx=0).stop 2023-11-24T06:16:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.339+0000 7f79548ac640 1 -- 172.21.15.5:0/946292306 shutdown_connections 2023-11-24T06:16:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.339+0000 7f79548ac640 1 --2- 172.21.15.5:0/946292306 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f793006ae40 0x7f793006d300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.339+0000 7f79548ac640 1 --2- 172.21.15.5:0/946292306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7950143950 0x7f795006c130 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.339+0000 7f79548ac640 1 --2- 172.21.15.5:0/946292306 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7950143570 0x7f79500cdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.339+0000 7f79548ac640 1 --2- 172.21.15.5:0/946292306 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f79500cf760 0x7f7950159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.340+0000 7f79548ac640 1 -- 172.21.15.5:0/946292306 >> 172.21.15.5:0/946292306 conn(0x7f795006a860 msgr2=0x7f79500b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:08.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.340+0000 7f79548ac640 1 -- 172.21.15.5:0/946292306 shutdown_connections 2023-11-24T06:16:08.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.340+0000 7f79548ac640 1 -- 172.21.15.5:0/946292306 wait complete. 2023-11-24T06:16:08.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:16:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.681+0000 7f9f8f543640 1 Processor -- start 2023-11-24T06:16:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.681+0000 7f9f8f543640 1 -- start start 2023-11-24T06:16:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.682+0000 7f9f8f543640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f9f88160970 0x7f9f880b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.682+0000 7f9f8f543640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9f880b3140 0x7f9f8806d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.682+0000 7f9f8f543640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9f88070780 0x7f9f8806da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.682+0000 7f9f8f543640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f88071cd0 con 0x7f9f88070780 2023-11-24T06:16:08.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.682+0000 7f9f8f543640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f880b2120 con 0x7f9f88160970 2023-11-24T06:16:08.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.682+0000 7f9f8f543640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f880b2400 con 0x7f9f880b3140 2023-11-24T06:16:08.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.683+0000 7f9f8dae0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9f88070780 0x7f9f8806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:08.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.683+0000 7f9f8dae0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9f88070780 0x7f9f8806da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41676/0 (socket says 172.21.15.5:41676) 2023-11-24T06:16:08.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.683+0000 7f9f8dae0640 1 -- 172.21.15.5:0/426472916 learned_addr learned my addr 172.21.15.5:0/426472916 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:08.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.683+0000 7f9f8d2df640 1 --2- 172.21.15.5:0/426472916 >> v2:172.21.15.6:3300/0 conn(0x7f9f88160970 0x7f9f880b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:08.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.683+0000 7f9f8cade640 1 --2- 172.21.15.5:0/426472916 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9f880b3140 0x7f9f8806d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:08.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.683+0000 7f9f8dae0640 1 -- 172.21.15.5:0/426472916 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9f880b3140 msgr2=0x7f9f8806d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:08.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.683+0000 7f9f8dae0640 1 --2- 172.21.15.5:0/426472916 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9f880b3140 0x7f9f8806d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.683+0000 7f9f8dae0640 1 -- 172.21.15.5:0/426472916 >> v2:172.21.15.6:3300/0 conn(0x7f9f88160970 msgr2=0x7f9f880b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:08.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.683+0000 7f9f8dae0640 1 --2- 172.21.15.5:0/426472916 >> v2:172.21.15.6:3300/0 conn(0x7f9f88160970 0x7f9f880b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.683+0000 7f9f8dae0640 1 -- 172.21.15.5:0/426472916 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f880b26e0 con 0x7f9f88070780 2023-11-24T06:16:08.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.685+0000 7f9f8dae0640 1 --2- 172.21.15.5:0/426472916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f88070780 0x7f9f8806da60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9f880b5210 tx=0x7f9f8002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c3faa261ea9eb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.685+0000 7f9f7ffff640 1 -- 172.21.15.5:0/426472916 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8002c9e0 con 0x7f9f88070780 2023-11-24T06:16:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.685+0000 7f9f7ffff640 1 -- 172.21.15.5:0/426472916 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8002cb40 con 0x7f9f88070780 2023-11-24T06:16:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.685+0000 7f9f7ffff640 1 -- 172.21.15.5:0/426472916 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f800343f0 con 0x7f9f88070780 2023-11-24T06:16:08.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.686+0000 7f9f8f543640 1 -- 172.21.15.5:0/426472916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f88070780 msgr2=0x7f9f8806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.686+0000 7f9f8f543640 1 --2- 172.21.15.5:0/426472916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f88070780 0x7f9f8806da60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f9f880b5210 tx=0x7f9f8002be40 comp rx=0 tx=0).stop 2023-11-24T06:16:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.686+0000 7f9f8f543640 1 -- 172.21.15.5:0/426472916 shutdown_connections 2023-11-24T06:16:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.686+0000 7f9f8f543640 1 --2- 172.21.15.5:0/426472916 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9f880b3140 0x7f9f8806d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.686+0000 7f9f8f543640 1 --2- 172.21.15.5:0/426472916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f88070780 0x7f9f8806da60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.686+0000 7f9f8f543640 1 --2- 172.21.15.5:0/426472916 >> v2:172.21.15.6:3300/0 conn(0x7f9f88160970 0x7f9f880b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.686+0000 7f9f8f543640 1 -- 172.21.15.5:0/426472916 >> 172.21.15.5:0/426472916 conn(0x7f9f880b0d80 msgr2=0x7f9f880b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:08.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.686+0000 7f9f8f543640 1 -- 172.21.15.5:0/426472916 shutdown_connections 2023-11-24T06:16:08.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.686+0000 7f9f8f543640 1 -- 172.21.15.5:0/426472916 wait complete. 2023-11-24T06:16:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.688+0000 7f9f8f543640 1 Processor -- start 2023-11-24T06:16:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.688+0000 7f9f8f543640 1 -- start start 2023-11-24T06:16:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.689+0000 7f9f8f543640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f88070780 0x7f9f881f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.689+0000 7f9f8f543640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f880b3140 0x7f9f88098790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.689+0000 7f9f8f543640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9f88160970 0x7f9f881532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.689+0000 7f9f8f543640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f88152b10 con 0x7f9f880b3140 2023-11-24T06:16:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.689+0000 7f9f8f543640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f881f6b00 con 0x7f9f88070780 2023-11-24T06:16:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.689+0000 7f9f8f543640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f881f6de0 con 0x7f9f88160970 2023-11-24T06:16:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.689+0000 7f9f8cade640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f880b3140 0x7f9f88098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.689+0000 7f9f8cade640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f880b3140 0x7f9f88098790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41692/0 (socket says 172.21.15.5:41692) 2023-11-24T06:16:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.689+0000 7f9f8cade640 1 -- 172.21.15.5:0/646795111 learned_addr learned my addr 172.21.15.5:0/646795111 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.690+0000 7f9f8cade640 1 -- 172.21.15.5:0/646795111 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9f88160970 msgr2=0x7f9f881532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.690+0000 7f9f8cade640 1 --2- 172.21.15.5:0/646795111 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9f88160970 0x7f9f881532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.690+0000 7f9f8cade640 1 -- 172.21.15.5:0/646795111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f88070780 msgr2=0x7f9f881f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.690+0000 7f9f8cade640 1 --2- 172.21.15.5:0/646795111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f88070780 0x7f9f881f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.690+0000 7f9f8cade640 1 -- 172.21.15.5:0/646795111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f80009b30 con 0x7f9f880b3140 2023-11-24T06:16:08.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.690+0000 7f9f8cade640 1 --2- 172.21.15.5:0/646795111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f880b3140 0x7f9f88098790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800d7d0 tx=0x7f9f7800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:08.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.690+0000 7f9f7dffb640 1 -- 172.21.15.5:0/646795111 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78004020 con 0x7f9f880b3140 2023-11-24T06:16:08.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.690+0000 7f9f7dffb640 1 -- 172.21.15.5:0/646795111 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f780041a0 con 0x7f9f880b3140 2023-11-24T06:16:08.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.690+0000 7f9f8f543640 1 -- 172.21.15.5:0/646795111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f881f70c0 con 0x7f9f880b3140 2023-11-24T06:16:08.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.690+0000 7f9f7dffb640 1 -- 172.21.15.5:0/646795111 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78004350 con 0x7f9f880b3140 2023-11-24T06:16:08.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.691+0000 7f9f8f543640 1 -- 172.21.15.5:0/646795111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f881f67f0 con 0x7f9f880b3140 2023-11-24T06:16:08.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.692+0000 7f9f8f543640 1 -- 172.21.15.5:0/646795111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f88074770 con 0x7f9f880b3140 2023-11-24T06:16:08.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.693+0000 7f9f7dffb640 1 -- 172.21.15.5:0/646795111 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f780096a0 con 0x7f9f880b3140 2023-11-24T06:16:08.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.694+0000 7f9f7dffb640 1 --2- 172.21.15.5:0/646795111 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f9f6c06ae40 0x7f9f6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:08.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.694+0000 7f9f7dffb640 1 -- 172.21.15.5:0/646795111 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f780899d0 con 0x7f9f880b3140 2023-11-24T06:16:08.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.694+0000 7f9f8d2df640 1 --2- 172.21.15.5:0/646795111 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f9f6c06ae40 0x7f9f6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:08.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.695+0000 7f9f8d2df640 1 --2- 172.21.15.5:0/646795111 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f9f6c06ae40 0x7f9f6c06d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9f84006fd0 tx=0x7f9f84008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.698+0000 7f9f7dffb640 1 -- 172.21.15.5:0/646795111 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9f780549e0 con 0x7f9f880b3140 2023-11-24T06:16:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.932+0000 7f9f8f543640 1 -- 172.21.15.5:0/646795111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f8806e290 con 0x7f9f880b3140 2023-11-24T06:16:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.935+0000 7f9f7dffb640 1 -- 172.21.15.5:0/646795111 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4895 (secure 0 0 0) 0x7f9f78058000 con 0x7f9f880b3140 2023-11-24T06:16:08.938 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:08.938 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4009460946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4009460946},{"type":"v1","addr":"172.21.15.5:6837","nonce":4009460946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2496926427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2496926427},{"type":"v1","addr":"172.21.15.5:6836","nonce":2496926427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3829754769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3829754769},{"type":"v1","addr":"172.21.15.6:6835","nonce":3829754769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:15:51.611541+0000","modified":"2023-11-24T06:15:54.664234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6833/2760317954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2760317954},{"type":"v1","addr":"172.21.15.6:6833","nonce":2760317954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:16:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2023-11-24T06:16:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.938+0000 7f9f8f543640 1 -- 172.21.15.5:0/646795111 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f9f6c06ae40 msgr2=0x7f9f6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.938+0000 7f9f8f543640 1 --2- 172.21.15.5:0/646795111 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f9f6c06ae40 0x7f9f6c06d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9f84006fd0 tx=0x7f9f84008040 comp rx=0 tx=0).stop 2023-11-24T06:16:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.938+0000 7f9f8f543640 1 -- 172.21.15.5:0/646795111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f880b3140 msgr2=0x7f9f88098790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.938+0000 7f9f8f543640 1 --2- 172.21.15.5:0/646795111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f880b3140 0x7f9f88098790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800d7d0 tx=0x7f9f7800dc90 comp rx=0 tx=0).stop 2023-11-24T06:16:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.938+0000 7f9f8f543640 1 -- 172.21.15.5:0/646795111 shutdown_connections 2023-11-24T06:16:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.938+0000 7f9f8f543640 1 --2- 172.21.15.5:0/646795111 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f9f6c06ae40 0x7f9f6c06d300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.938+0000 7f9f8f543640 1 --2- 172.21.15.5:0/646795111 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9f88160970 0x7f9f881532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.938+0000 7f9f8f543640 1 --2- 172.21.15.5:0/646795111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f880b3140 0x7f9f88098790 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.938+0000 7f9f8f543640 1 --2- 172.21.15.5:0/646795111 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9f88070780 0x7f9f881f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.938+0000 7f9f8f543640 1 -- 172.21.15.5:0/646795111 >> 172.21.15.5:0/646795111 conn(0x7f9f880b0d80 msgr2=0x7f9f8806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.938+0000 7f9f8f543640 1 -- 172.21.15.5:0/646795111 shutdown_connections 2023-11-24T06:16:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:08.939+0000 7f9f8f543640 1 -- 172.21.15.5:0/646795111 wait complete. 2023-11-24T06:16:09.364 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:15:51.611541+0000', 'modified': '2023-11-24T06:15:54.664234+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.6:6833/2760317954', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.6:6832', 'nonce': 2760317954}, {'type': 'v1', 'addr': '172.21.15.6:6833', 'nonce': 2760317954}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:16:09.365 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:16:09.365 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-24T06:16:09.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.600+0000 7f755f147640 1 Processor -- start 2023-11-24T06:16:09.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.600+0000 7f755f147640 1 -- start start 2023-11-24T06:16:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.602+0000 7f755f147640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7558071a00 0x7f755815e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.602+0000 7f755f147640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7558070e90 0x7f75580ce130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.602+0000 7f755f147640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75580ce670 con 0x7f7558071620 2023-11-24T06:16:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.602+0000 7f755f147640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75580ce8d0 con 0x7f7558070e90 2023-11-24T06:16:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.602+0000 7f755f147640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75580700d0 con 0x7f7558071a00 2023-11-24T06:16:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.602+0000 7f755d944640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7558071a00 0x7f755815e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.602+0000 7f755e946640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7558070e90 0x7f75580ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.603+0000 7f755d944640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7558071a00 0x7f755815e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:35494/0 (socket says 172.21.15.5:35494) 2023-11-24T06:16:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.603+0000 7f755d944640 1 -- 172.21.15.5:0/178015360 learned_addr learned my addr 172.21.15.5:0/178015360 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.603+0000 7f755d944640 1 -- 172.21.15.5:0/178015360 >> v2:172.21.15.6:3300/0 conn(0x7f7558070e90 msgr2=0x7f75580ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.603+0000 7f755d944640 1 --2- 172.21.15.5:0/178015360 >> v2:172.21.15.6:3300/0 conn(0x7f7558070e90 0x7f75580ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.603+0000 7f755d944640 1 -- 172.21.15.5:0/178015360 >> v1:172.21.15.6:6789/0 conn(0x7f7558071620 legacy=0x7f75580cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.603+0000 7f755d944640 1 -- 172.21.15.5:0/178015360 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75580703b0 con 0x7f7558071a00 2023-11-24T06:16:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.603+0000 7f755e946640 1 --2- 172.21.15.5:0/178015360 >> v2:172.21.15.6:3300/0 conn(0x7f7558070e90 0x7f75580ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:16:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.605+0000 7f755d944640 1 --2- 172.21.15.5:0/178015360 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7558071a00 0x7f755815e7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7548009870 tx=0x7f754802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9de9df63d2ed2c6e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.605+0000 7f755d143640 1 -- 172.21.15.5:0/178015360 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548031640 con 0x7f7558071a00 2023-11-24T06:16:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.606+0000 7f755d143640 1 -- 172.21.15.5:0/178015360 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75480317a0 con 0x7f7558071a00 2023-11-24T06:16:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.606+0000 7f755d143640 1 -- 172.21.15.5:0/178015360 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548031b30 con 0x7f7558071a00 2023-11-24T06:16:09.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.606+0000 7f755f147640 1 -- 172.21.15.5:0/178015360 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7558071a00 msgr2=0x7f755815e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:09.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.606+0000 7f755f147640 1 --2- 172.21.15.5:0/178015360 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7558071a00 0x7f755815e7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7548009870 tx=0x7f754802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.606+0000 7f755f147640 1 -- 172.21.15.5:0/178015360 shutdown_connections 2023-11-24T06:16:09.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.606+0000 7f755f147640 1 --2- 172.21.15.5:0/178015360 >> v2:172.21.15.6:3300/0 conn(0x7f7558070e90 0x7f75580ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.606+0000 7f755f147640 1 --2- 172.21.15.5:0/178015360 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7558071a00 0x7f755815e7f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.606+0000 7f755f147640 1 -- 172.21.15.5:0/178015360 >> 172.21.15.5:0/178015360 conn(0x7f75580b0840 msgr2=0x7f75580b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:09.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.607+0000 7f755f147640 1 -- 172.21.15.5:0/178015360 shutdown_connections 2023-11-24T06:16:09.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.607+0000 7f755f147640 1 -- 172.21.15.5:0/178015360 wait complete. 2023-11-24T06:16:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.608+0000 7f755f147640 1 Processor -- start 2023-11-24T06:16:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.608+0000 7f755f147640 1 -- start start 2023-11-24T06:16:09.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.609+0000 7f755f147640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7558070e90 0x7f75580c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.609+0000 7f755f147640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7558071620 0x7f75580b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.609+0000 7f755f147640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7558071a00 0x7f7558151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.609+0000 7f755f147640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75580b37c0 con 0x7f7558071a00 2023-11-24T06:16:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.609+0000 7f755f147640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75580c41b0 con 0x7f7558070e90 2023-11-24T06:16:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.609+0000 7f755f147640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75580c4490 con 0x7f7558071620 2023-11-24T06:16:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.610+0000 7f755e946640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7558071a00 0x7f7558151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.610+0000 7f755e946640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7558071a00 0x7f7558151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41708/0 (socket says 172.21.15.5:41708) 2023-11-24T06:16:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.610+0000 7f755e946640 1 -- 172.21.15.5:0/4112072348 learned_addr learned my addr 172.21.15.5:0/4112072348 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:09.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.610+0000 7f755d944640 1 --2- 172.21.15.5:0/4112072348 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7558071620 0x7f75580b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.610+0000 7f755e145640 1 --2- 172.21.15.5:0/4112072348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7558070e90 0x7f75580c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.610+0000 7f755e946640 1 -- 172.21.15.5:0/4112072348 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7558071620 msgr2=0x7f75580b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:09.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.610+0000 7f755e946640 1 --2- 172.21.15.5:0/4112072348 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7558071620 0x7f75580b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.610+0000 7f755e946640 1 -- 172.21.15.5:0/4112072348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7558070e90 msgr2=0x7f75580c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:09.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.610+0000 7f755e946640 1 --2- 172.21.15.5:0/4112072348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7558070e90 0x7f75580c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.610+0000 7f755e946640 1 -- 172.21.15.5:0/4112072348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7550009400 con 0x7f7558071a00 2023-11-24T06:16:09.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.611+0000 7f755e946640 1 --2- 172.21.15.5:0/4112072348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7558071a00 0x7f7558151860 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f755000e930 tx=0x7f755000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:09.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.611+0000 7f754effd640 1 -- 172.21.15.5:0/4112072348 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550009700 con 0x7f7558071a00 2023-11-24T06:16:09.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.611+0000 7f754effd640 1 -- 172.21.15.5:0/4112072348 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755000ca00 con 0x7f7558071a00 2023-11-24T06:16:09.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.611+0000 7f754effd640 1 -- 172.21.15.5:0/4112072348 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755000cd90 con 0x7f7558071a00 2023-11-24T06:16:09.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.611+0000 7f755f147640 1 -- 172.21.15.5:0/4112072348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75480094d0 con 0x7f7558071a00 2023-11-24T06:16:09.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.611+0000 7f755f147640 1 -- 172.21.15.5:0/4112072348 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75580c4a50 con 0x7f7558071a00 2023-11-24T06:16:09.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.613+0000 7f754effd640 1 -- 172.21.15.5:0/4112072348 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7550013470 con 0x7f7558071a00 2023-11-24T06:16:09.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.614+0000 7f754effd640 1 --2- 172.21.15.5:0/4112072348 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f753c06ae40 0x7f753c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.615+0000 7f754effd640 1 -- 172.21.15.5:0/4112072348 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7550089340 con 0x7f7558071a00 2023-11-24T06:16:09.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.615+0000 7f755e145640 1 --2- 172.21.15.5:0/4112072348 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f753c06ae40 0x7f753c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.615+0000 7f7563015740 1 Processor -- start 2023-11-24T06:16:09.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.615+0000 7f7563015740 1 -- start start 2023-11-24T06:16:09.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.615+0000 7f755e145640 1 --2- 172.21.15.5:0/4112072348 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f753c06ae40 0x7f753c06d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f75540012e0 tx=0x7f7554009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.616+0000 7f7563015740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb4545070 0x55cdb4545450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.616+0000 7f7563015740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 0x55cdb4544270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.616+0000 7f7563015740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb45447b0 0x55cdb4588730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.616+0000 7f7563015740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb44a3880 con 0x55cdb4545070 2023-11-24T06:16:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.616+0000 7f7563015740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb44b2ae0 con 0x55cdb45447b0 2023-11-24T06:16:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.616+0000 7f7563015740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdb44b01d0 con 0x55cdb453fd70 2023-11-24T06:16:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.616+0000 7f755e145640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb4545070 0x55cdb4545450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.616+0000 7f755e145640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb4545070 0x55cdb4545450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41718/0 (socket says 172.21.15.5:41718) 2023-11-24T06:16:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.616+0000 7f755e145640 1 -- 172.21.15.5:0/1316953451 learned_addr learned my addr 172.21.15.5:0/1316953451 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.617+0000 7f755e145640 1 -- 172.21.15.5:0/1316953451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 msgr2=0x55cdb4544270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.617+0000 7f755e145640 1 --2- 172.21.15.5:0/1316953451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 0x55cdb4544270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.617+0000 7f755e946640 1 --2- 172.21.15.5:0/1316953451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb45447b0 0x55cdb4588730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.617+0000 7f755e145640 1 -- 172.21.15.5:0/1316953451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb45447b0 msgr2=0x55cdb4588730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:09.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.617+0000 7f755e145640 1 --2- 172.21.15.5:0/1316953451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb45447b0 0x55cdb4588730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.617+0000 7f755e145640 1 -- 172.21.15.5:0/1316953451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdb4473860 con 0x55cdb4545070 2023-11-24T06:16:09.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.617+0000 7f755e145640 1 --2- 172.21.15.5:0/1316953451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb4545070 0x55cdb4545450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7554003b90 tx=0x7f7554011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7ef5977e5fe433 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:09.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.617+0000 7f753bfff640 1 -- 172.21.15.5:0/1316953451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75540052e0 con 0x55cdb4545070 2023-11-24T06:16:09.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.617+0000 7f753bfff640 1 -- 172.21.15.5:0/1316953451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7554005440 con 0x55cdb4545070 2023-11-24T06:16:09.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.617+0000 7f753bfff640 1 -- 172.21.15.5:0/1316953451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75540057b0 con 0x55cdb4545070 2023-11-24T06:16:09.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.618+0000 7f7563015740 1 -- 172.21.15.5:0/1316953451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb4545070 msgr2=0x55cdb4545450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:09.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.618+0000 7f7563015740 1 --2- 172.21.15.5:0/1316953451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb4545070 0x55cdb4545450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7554003b90 tx=0x7f7554011000 comp rx=0 tx=0).stop 2023-11-24T06:16:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.618+0000 7f7563015740 1 -- 172.21.15.5:0/1316953451 shutdown_connections 2023-11-24T06:16:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.618+0000 7f7563015740 1 --2- 172.21.15.5:0/1316953451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 0x55cdb4544270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.618+0000 7f7563015740 1 --2- 172.21.15.5:0/1316953451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb45447b0 0x55cdb4588730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.618+0000 7f7563015740 1 --2- 172.21.15.5:0/1316953451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb4545070 0x55cdb4545450 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.618+0000 7f7563015740 1 -- 172.21.15.5:0/1316953451 >> 172.21.15.5:0/1316953451 conn(0x55cdb4506c70 msgr2=0x55cdb458ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.618+0000 7f7563015740 1 -- 172.21.15.5:0/1316953451 shutdown_connections 2023-11-24T06:16:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.618+0000 7f7563015740 1 -- 172.21.15.5:0/1316953451 wait complete. 2023-11-24T06:16:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.619+0000 7f7563015740 1 -- start start 2023-11-24T06:16:09.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.619+0000 7f7563015740 1 Processor -- start 2023-11-24T06:16:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.620+0000 7f7563015740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 0x55cdb45694f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.620+0000 7f7563015740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb45447b0 0x55cdb456d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.620+0000 7f7563015740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb4545070 0x55cdb4590c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.620+0000 7f7563015740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb44abac0 con 0x55cdb45447b0 2023-11-24T06:16:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.620+0000 7f7563015740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb44482f0 con 0x55cdb4545070 2023-11-24T06:16:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.620+0000 7f7563015740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdb4465c70 con 0x55cdb453fd70 2023-11-24T06:16:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.620+0000 7f7563015740 10 client.0 _ll_register_callbacks cb 0x55cdb446a3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:16:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.620+0000 7f755d944640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb45447b0 0x55cdb456d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.620+0000 7f755d944640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb45447b0 0x55cdb456d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41728/0 (socket says 172.21.15.5:41728) 2023-11-24T06:16:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.620+0000 7f755d944640 1 -- 172.21.15.5:0/2844634302 learned_addr learned my addr 172.21.15.5:0/2844634302 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.621+0000 7f755e145640 1 --2- 172.21.15.5:0/2844634302 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 0x55cdb45694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.621+0000 7f755e946640 1 --2- 172.21.15.5:0/2844634302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb4545070 0x55cdb4590c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.621+0000 7f755d944640 1 -- 172.21.15.5:0/2844634302 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 msgr2=0x55cdb45694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.621+0000 7f755d944640 1 --2- 172.21.15.5:0/2844634302 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 0x55cdb45694f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.621+0000 7f755d944640 1 -- 172.21.15.5:0/2844634302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb4545070 msgr2=0x55cdb4590c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.621+0000 7f755d944640 1 --2- 172.21.15.5:0/2844634302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb4545070 0x55cdb4590c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.621+0000 7f755d944640 1 -- 172.21.15.5:0/2844634302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755400f040 con 0x55cdb45447b0 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.621+0000 7f755e946640 1 --2- 172.21.15.5:0/2844634302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb4545070 0x55cdb4590c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.621+0000 7f755d944640 1 --2- 172.21.15.5:0/2844634302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb45447b0 0x55cdb456d470 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7548000c00 tx=0x7f7548004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eaf7f074cbc41db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.621+0000 7f75397fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.622+0000 7f75397fa640 1 -- 172.21.15.5:0/2844634302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548045680 con 0x55cdb45447b0 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.622+0000 7f75397fa640 1 -- 172.21.15.5:0/2844634302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f754803f070 con 0x55cdb45447b0 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.622+0000 7f7563015740 1 -- 172.21.15.5:0/2844634302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdb44a4aa0 con 0x55cdb45447b0 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.622+0000 7f75397fa640 1 -- 172.21.15.5:0/2844634302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75480457e0 con 0x55cdb45447b0 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.622+0000 7f7563015740 10 client.4632 fetch_fsmap learned FSMap version 8 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.622+0000 7f7563015740 1 -- 172.21.15.5:0/2844634302 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55cdb44aa140 con 0x55cdb45447b0 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.622+0000 7f75397fa640 1 -- 172.21.15.5:0/2844634302 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cdb44a4aa0 con 0x55cdb45447b0 2023-11-24T06:16:09.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.622+0000 7f75397fa640 1 -- 172.21.15.5:0/2844634302 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7548045940 con 0x55cdb45447b0 2023-11-24T06:16:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.622+0000 7f7563015740 10 client.4632 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T06:16:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.622+0000 7f7563015740 10 client.4632 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-24T06:16:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.622+0000 7f7563015740 20 client.4632 populate_metadata read hostname 'smithi005' 2023-11-24T06:16:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.623+0000 7f7563015740 1 --2- 172.21.15.5:0/2844634302 >> [v2:172.21.15.6:6832/2760317954,v1:172.21.15.6:6833/2760317954] conn(0x55cdb458e170 0x55cdb459dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.623+0000 7f7563015740 4 client.4632 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:16:09.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.623+0000 7f7563015740 1 -- 172.21.15.5:0/2844634302 --> [v2:172.21.15.6:6832/2760317954,v1:172.21.15.6:6833/2760317954] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdb44a47c0 con 0x55cdb458e170 2023-11-24T06:16:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.623+0000 7f755e145640 1 --2- 172.21.15.5:0/2844634302 >> [v2:172.21.15.6:6832/2760317954,v1:172.21.15.6:6833/2760317954] conn(0x55cdb458e170 0x55cdb459dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.624+0000 7f755e145640 1 --2- 172.21.15.5:0/2844634302 >> [v2:172.21.15.6:6832/2760317954,v1:172.21.15.6:6833/2760317954] conn(0x55cdb458e170 0x55cdb459dbd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7391d95e8a65ef9b server_cookie=f461f242434bdb89 in_seq=0 out_seq=0 2023-11-24T06:16:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.624+0000 7f75397fa640 10 client.4632 ms_handle_connect on v2:172.21.15.6:6832/2760317954 2023-11-24T06:16:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.626+0000 7f75397fa640 1 -- 172.21.15.5:0/2844634302 <== mds.0 v2:172.21.15.6:6832/2760317954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cdb44a47c0 con 0x55cdb458e170 2023-11-24T06:16:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.626+0000 7f75397fa640 10 client.4632 handle_command_reply: tid=0 2023-11-24T06:16:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.626+0000 7f7563015740 1 client.4632 shutdown 2023-11-24T06:16:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.627+0000 7f7563015740 1 -- 172.21.15.5:0/2844634302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb45447b0 msgr2=0x55cdb456d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.627+0000 7f7563015740 1 --2- 172.21.15.5:0/2844634302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb45447b0 0x55cdb456d470 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7548000c00 tx=0x7f7548004730 comp rx=0 tx=0).stop 2023-11-24T06:16:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.627+0000 7f7563015740 1 -- 172.21.15.5:0/2844634302 shutdown_connections 2023-11-24T06:16:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.627+0000 7f7563015740 1 --2- 172.21.15.5:0/2844634302 >> [v2:172.21.15.6:6832/2760317954,v1:172.21.15.6:6833/2760317954] conn(0x55cdb458e170 0x55cdb459dbd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.627+0000 7f7563015740 1 --2- 172.21.15.5:0/2844634302 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb4545070 0x55cdb4590c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.627+0000 7f7563015740 1 --2- 172.21.15.5:0/2844634302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb45447b0 0x55cdb456d470 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.627+0000 7f75397fa640 0 client.4632 ms_handle_reset on v2:172.21.15.6:6832/2760317954 2023-11-24T06:16:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.627+0000 7f7563015740 1 --2- 172.21.15.5:0/2844634302 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 0x55cdb45694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.627+0000 7f7563015740 1 -- 172.21.15.5:0/2844634302 >> 172.21.15.5:0/2844634302 conn(0x55cdb4506c70 msgr2=0x55cdb4589f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.627+0000 7f7563015740 1 -- 172.21.15.5:0/2844634302 shutdown_connections 2023-11-24T06:16:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.627+0000 7f7563015740 1 -- 172.21.15.5:0/2844634302 wait complete. 2023-11-24T06:16:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.627+0000 7f7563015740 20 client.4632 trim_cache size 0 max 16384 2023-11-24T06:16:09.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.646+0000 7f7563015740 1 Processor -- start 2023-11-24T06:16:09.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.646+0000 7f7563015740 1 -- start start 2023-11-24T06:16:09.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.647+0000 7f7563015740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 0x55cdb4569300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.647+0000 7f7563015740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb45447b0 0x55cdb4589dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.647+0000 7f7563015740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb4545070 0x55cdb453edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.647+0000 7f7563015740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb420ef20 con 0x55cdb45447b0 2023-11-24T06:16:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.647+0000 7f7563015740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb4591b40 con 0x55cdb4545070 2023-11-24T06:16:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.647+0000 7f7563015740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdb45ad850 con 0x55cdb453fd70 2023-11-24T06:16:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.647+0000 7f755d944640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb45447b0 0x55cdb4589dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.647+0000 7f755d944640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb45447b0 0x55cdb4589dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41730/0 (socket says 172.21.15.5:41730) 2023-11-24T06:16:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.647+0000 7f755d944640 1 -- 172.21.15.5:0/1671907950 learned_addr learned my addr 172.21.15.5:0/1671907950 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.647+0000 7f755d944640 1 -- 172.21.15.5:0/1671907950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 msgr2=0x55cdb4569300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.647+0000 7f755d944640 1 --2- 172.21.15.5:0/1671907950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 0x55cdb4569300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.647+0000 7f755e946640 1 --2- 172.21.15.5:0/1671907950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb4545070 0x55cdb453edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.647+0000 7f755d944640 1 -- 172.21.15.5:0/1671907950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb4545070 msgr2=0x55cdb453edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.648+0000 7f755d944640 1 --2- 172.21.15.5:0/1671907950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb4545070 0x55cdb453edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.648+0000 7f755d944640 1 -- 172.21.15.5:0/1671907950 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdb4447b80 con 0x55cdb45447b0 2023-11-24T06:16:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.648+0000 7f755d944640 1 --2- 172.21.15.5:0/1671907950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb45447b0 0x55cdb4589dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f75480090b0 tx=0x7f754803f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a52146c1c82ac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.648+0000 7f754cdff640 1 -- 172.21.15.5:0/1671907950 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548045ea0 con 0x55cdb45447b0 2023-11-24T06:16:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.648+0000 7f754cdff640 1 -- 172.21.15.5:0/1671907950 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7548004240 con 0x55cdb45447b0 2023-11-24T06:16:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.648+0000 7f754cdff640 1 -- 172.21.15.5:0/1671907950 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7548045480 con 0x55cdb45447b0 2023-11-24T06:16:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.649+0000 7f7563015740 1 -- 172.21.15.5:0/1671907950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb45447b0 msgr2=0x55cdb4589dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.649+0000 7f7563015740 1 --2- 172.21.15.5:0/1671907950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb45447b0 0x55cdb4589dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f75480090b0 tx=0x7f754803f000 comp rx=0 tx=0).stop 2023-11-24T06:16:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.649+0000 7f7563015740 1 -- 172.21.15.5:0/1671907950 shutdown_connections 2023-11-24T06:16:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.649+0000 7f7563015740 1 --2- 172.21.15.5:0/1671907950 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb4545070 0x55cdb453edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.649+0000 7f7563015740 1 --2- 172.21.15.5:0/1671907950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb45447b0 0x55cdb4589dd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.649+0000 7f7563015740 1 --2- 172.21.15.5:0/1671907950 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 0x55cdb4569300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.649+0000 7f7563015740 1 -- 172.21.15.5:0/1671907950 >> 172.21.15.5:0/1671907950 conn(0x55cdb4506c70 msgr2=0x55cdb4568960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.649+0000 7f7563015740 1 -- 172.21.15.5:0/1671907950 shutdown_connections 2023-11-24T06:16:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.649+0000 7f7563015740 1 -- 172.21.15.5:0/1671907950 wait complete. 2023-11-24T06:16:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.650+0000 7f7563015740 1 -- start start 2023-11-24T06:16:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.650+0000 7f7563015740 1 Processor -- start 2023-11-24T06:16:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.651+0000 7f7563015740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 0x55cdb458fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.651+0000 7f7563015740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb45447b0 0x55cdb456f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.651+0000 7f7563015740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb4545070 0x55cdb45a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.651+0000 7f7563015740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb4543fe0 con 0x55cdb4545070 2023-11-24T06:16:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.651+0000 7f7563015740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdb456e450 con 0x55cdb45447b0 2023-11-24T06:16:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.651+0000 7f7563015740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdb45f8460 con 0x55cdb453fd70 2023-11-24T06:16:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.652+0000 7f7563015740 10 client.0 _ll_register_callbacks cb 0x7f7550009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:16:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.652+0000 7f755e946640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb4545070 0x55cdb45a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.652+0000 7f755e946640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb4545070 0x55cdb45a3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41746/0 (socket says 172.21.15.5:41746) 2023-11-24T06:16:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.652+0000 7f755e946640 1 -- 172.21.15.5:0/1565465886 learned_addr learned my addr 172.21.15.5:0/1565465886 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.652+0000 7f755e145640 1 --2- 172.21.15.5:0/1565465886 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 0x55cdb458fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.652+0000 7f755e946640 1 -- 172.21.15.5:0/1565465886 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 msgr2=0x55cdb458fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.652+0000 7f755e946640 1 --2- 172.21.15.5:0/1565465886 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 0x55cdb458fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.652+0000 7f755d944640 1 --2- 172.21.15.5:0/1565465886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb45447b0 0x55cdb456f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.652+0000 7f755e946640 1 -- 172.21.15.5:0/1565465886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb45447b0 msgr2=0x55cdb456f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.652+0000 7f755e946640 1 --2- 172.21.15.5:0/1565465886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb45447b0 0x55cdb456f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.652+0000 7f755e946640 1 -- 172.21.15.5:0/1565465886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdb44aa140 con 0x55cdb4545070 2023-11-24T06:16:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.652+0000 7f755e946640 1 --2- 172.21.15.5:0/1565465886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb4545070 0x55cdb45a3d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7550057dd0 tx=0x7f7550058190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ab195c79700638 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.652+0000 7f75397fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-11-24T06:16:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.653+0000 7f75397fa640 1 -- 172.21.15.5:0/1565465886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550058330 con 0x55cdb4545070 2023-11-24T06:16:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.653+0000 7f75397fa640 1 -- 172.21.15.5:0/1565465886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7550058490 con 0x55cdb4545070 2023-11-24T06:16:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.653+0000 7f7563015740 1 -- 172.21.15.5:0/1565465886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7554005440 con 0x55cdb4545070 2023-11-24T06:16:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.653+0000 7f75397fa640 1 -- 172.21.15.5:0/1565465886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75500575c0 con 0x55cdb4545070 2023-11-24T06:16:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.653+0000 7f7563015740 10 client.4638 fetch_fsmap learned FSMap version 8 2023-11-24T06:16:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.653+0000 7f7563015740 1 -- 172.21.15.5:0/1565465886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55cdb446d170 con 0x55cdb4545070 2023-11-24T06:16:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.653+0000 7f75397fa640 1 -- 172.21.15.5:0/1565465886 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7554005440 con 0x55cdb4545070 2023-11-24T06:16:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.653+0000 7f75397fa640 1 -- 172.21.15.5:0/1565465886 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7550011070 con 0x55cdb4545070 2023-11-24T06:16:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.653+0000 7f7563015740 10 client.4638 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T06:16:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.653+0000 7f7563015740 10 client.4638 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-24T06:16:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.653+0000 7f7563015740 20 client.4638 populate_metadata read hostname 'smithi005' 2023-11-24T06:16:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.653+0000 7f7563015740 1 --2- 172.21.15.5:0/1565465886 >> [v2:172.21.15.6:6832/2760317954,v1:172.21.15.6:6833/2760317954] conn(0x55cdb4572850 0x55cdb4572c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.653+0000 7f7563015740 4 client.4638 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:16:09.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.653+0000 7f7563015740 1 -- 172.21.15.5:0/1565465886 --> [v2:172.21.15.6:6832/2760317954,v1:172.21.15.6:6833/2760317954] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cdb41ac6c0 con 0x55cdb4572850 2023-11-24T06:16:09.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.654+0000 7f755e145640 1 --2- 172.21.15.5:0/1565465886 >> [v2:172.21.15.6:6832/2760317954,v1:172.21.15.6:6833/2760317954] conn(0x55cdb4572850 0x55cdb4572c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.655+0000 7f755e145640 1 --2- 172.21.15.5:0/1565465886 >> [v2:172.21.15.6:6832/2760317954,v1:172.21.15.6:6833/2760317954] conn(0x55cdb4572850 0x55cdb4572c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d058a8d09806e3f8 server_cookie=dfeee2c2f91acb8e in_seq=0 out_seq=0 2023-11-24T06:16:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.655+0000 7f75397fa640 10 client.4638 ms_handle_connect on v2:172.21.15.6:6832/2760317954 2023-11-24T06:16:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f75397fa640 1 -- 172.21.15.5:0/1565465886 <== mds.0 v2:172.21.15.6:6832/2760317954 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55cdb41ac6c0 con 0x55cdb4572850 2023-11-24T06:16:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f75397fa640 10 client.4638 handle_command_reply: tid=0 2023-11-24T06:16:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f7563015740 1 client.4638 shutdown 2023-11-24T06:16:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f7563015740 1 -- 172.21.15.5:0/1565465886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb4545070 msgr2=0x55cdb45a3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f7563015740 1 --2- 172.21.15.5:0/1565465886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb4545070 0x55cdb45a3d10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7550057dd0 tx=0x7f7550058190 comp rx=0 tx=0).stop 2023-11-24T06:16:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f7563015740 1 -- 172.21.15.5:0/1565465886 shutdown_connections 2023-11-24T06:16:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f7563015740 1 --2- 172.21.15.5:0/1565465886 >> [v2:172.21.15.6:6832/2760317954,v1:172.21.15.6:6833/2760317954] conn(0x55cdb4572850 0x55cdb4572c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f7563015740 1 --2- 172.21.15.5:0/1565465886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdb4545070 0x55cdb45a3d10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f7563015740 1 --2- 172.21.15.5:0/1565465886 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x55cdb45447b0 0x55cdb456f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f7563015740 1 --2- 172.21.15.5:0/1565465886 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x55cdb453fd70 0x55cdb458fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f75397fa640 0 client.4638 ms_handle_reset on v2:172.21.15.6:6832/2760317954 2023-11-24T06:16:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f7563015740 1 -- 172.21.15.5:0/1565465886 >> 172.21.15.5:0/1565465886 conn(0x55cdb4506c70 msgr2=0x55cdb4589dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f7563015740 1 -- 172.21.15.5:0/1565465886 shutdown_connections 2023-11-24T06:16:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f7563015740 1 -- 172.21.15.5:0/1565465886 wait complete. 2023-11-24T06:16:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.656+0000 7f7563015740 20 client.4638 trim_cache size 0 max 16384 2023-11-24T06:16:09.662 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-11-24T06:16:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c", 2023-11-24T06:16:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2023-11-24T06:16:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4285, 2023-11-24T06:16:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-11-24T06:16:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-11-24T06:16:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-11-24T06:16:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 17.524603677999998, 2023-11-24T06:16:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 8, 2023-11-24T06:16:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 22, 2023-11-24T06:16:09.663 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 22, 2023-11-24T06:16:09.663 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 22.878479393999999 2023-11-24T06:16:09.663 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-11-24T06:16:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.657+0000 7f755f147640 1 -- 172.21.15.5:0/4112072348 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f753c06ae40 msgr2=0x7f753c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.657+0000 7f755f147640 1 --2- 172.21.15.5:0/4112072348 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f753c06ae40 0x7f753c06d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f75540012e0 tx=0x7f7554009040 comp rx=0 tx=0).stop 2023-11-24T06:16:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.657+0000 7f755f147640 1 -- 172.21.15.5:0/4112072348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7558071a00 msgr2=0x7f7558151860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.657+0000 7f755f147640 1 --2- 172.21.15.5:0/4112072348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7558071a00 0x7f7558151860 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f755000e930 tx=0x7f755000edf0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.657+0000 7f755f147640 1 -- 172.21.15.5:0/4112072348 shutdown_connections 2023-11-24T06:16:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.657+0000 7f755f147640 1 --2- 172.21.15.5:0/4112072348 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f753c06ae40 0x7f753c06d300 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.657+0000 7f755f147640 1 --2- 172.21.15.5:0/4112072348 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7558071620 0x7f75580b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.657+0000 7f755f147640 1 --2- 172.21.15.5:0/4112072348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7558071a00 0x7f7558151860 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.657+0000 7f755f147640 1 --2- 172.21.15.5:0/4112072348 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7558070e90 0x7f75580c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.657+0000 7f755f147640 1 -- 172.21.15.5:0/4112072348 >> 172.21.15.5:0/4112072348 conn(0x7f75580b0840 msgr2=0x7f7558152b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.657+0000 7f755f147640 1 -- 172.21.15.5:0/4112072348 shutdown_connections 2023-11-24T06:16:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:09.657+0000 7f755f147640 1 -- 172.21.15.5:0/4112072348 wait complete. 2023-11-24T06:16:09.677 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-24T06:16:09.701 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-24T06:16:09.701 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-24T06:16:09.701 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-24T06:16:09.702 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:16:09.702 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:16:09.702 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-24T06:16:09.702 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:16:09.702 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T06:16:09.702 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-24T06:16:09.702 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:16:09.702 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:16:09.702 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-24T06:16:09.702 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T06:16:09.702 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T06:16:09.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:09.703 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2023-11-24T06:16:09.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:09.762 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2023-11-24T06:16:09.879 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2023-11-24T06:16:09.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:16:09.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:09.884 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-24T06:16:09.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:09.947 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2023-11-24T06:16:10.052 INFO:teuthology.orchestra.run.smithi006.stderr:Cannot find device "ceph-brx" 2023-11-24T06:16:10.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:16:10.056 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-24T06:16:10.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:16:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.285+0000 7f2fea251640 1 Processor -- start 2023-11-24T06:16:10.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.285+0000 7f2fea251640 1 -- start start 2023-11-24T06:16:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.287+0000 7f2fea251640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2fe415aea0 0x7f2fe4141280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.287+0000 7f2fea251640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2fe415b280 0x7f2fe406eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.287+0000 7f2fea251640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe40cfa00 con 0x7f2fe406f040 2023-11-24T06:16:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.287+0000 7f2fea251640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe406f470 con 0x7f2fe415aea0 2023-11-24T06:16:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.287+0000 7f2fea251640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe415fc20 con 0x7f2fe415b280 2023-11-24T06:16:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.287+0000 7f2fe37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2fe415aea0 0x7f2fe4141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.287+0000 7f2fe37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2fe415aea0 0x7f2fe4141280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41760/0 (socket says 172.21.15.5:41760) 2023-11-24T06:16:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.287+0000 7f2fe37fe640 1 -- 172.21.15.5:0/4134640437 learned_addr learned my addr 172.21.15.5:0/4134640437 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.288+0000 7f2fdbfff640 1 --2- 172.21.15.5:0/4134640437 >> v2:172.21.15.6:3300/0 conn(0x7f2fe415b280 0x7f2fe406eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.288+0000 7f2fe37fe640 1 -- 172.21.15.5:0/4134640437 >> v2:172.21.15.6:3300/0 conn(0x7f2fe415b280 msgr2=0x7f2fe406eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.288+0000 7f2fe37fe640 1 --2- 172.21.15.5:0/4134640437 >> v2:172.21.15.6:3300/0 conn(0x7f2fe415b280 0x7f2fe406eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.288+0000 7f2fe37fe640 1 -- 172.21.15.5:0/4134640437 >> v1:172.21.15.5:6789/0 conn(0x7f2fe406f040 legacy=0x7f2fe40d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.288+0000 7f2fe37fe640 1 -- 172.21.15.5:0/4134640437 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe41417c0 con 0x7f2fe415aea0 2023-11-24T06:16:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.288+0000 7f2fdbfff640 1 --2- 172.21.15.5:0/4134640437 >> v2:172.21.15.6:3300/0 conn(0x7f2fe415b280 0x7f2fe406eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.289+0000 7f2fe37fe640 1 --2- 172.21.15.5:0/4134640437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe415aea0 0x7f2fe4141280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40704a0 tx=0x7f2fd402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c06954bb84ed36f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.289+0000 7f2fe2ffd640 1 -- 172.21.15.5:0/4134640437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4030860 con 0x7f2fe415aea0 2023-11-24T06:16:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.289+0000 7f2fe2ffd640 1 -- 172.21.15.5:0/4134640437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd40309c0 con 0x7f2fe415aea0 2023-11-24T06:16:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.289+0000 7f2fe2ffd640 1 -- 172.21.15.5:0/4134640437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4030d50 con 0x7f2fe415aea0 2023-11-24T06:16:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.290+0000 7f2fea251640 1 -- 172.21.15.5:0/4134640437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe415aea0 msgr2=0x7f2fe4141280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.290+0000 7f2fea251640 1 --2- 172.21.15.5:0/4134640437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe415aea0 0x7f2fe4141280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40704a0 tx=0x7f2fd402b9a0 comp rx=0 tx=0).stop 2023-11-24T06:16:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.290+0000 7f2fea251640 1 -- 172.21.15.5:0/4134640437 shutdown_connections 2023-11-24T06:16:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.290+0000 7f2fea251640 1 --2- 172.21.15.5:0/4134640437 >> v2:172.21.15.6:3300/0 conn(0x7f2fe415b280 0x7f2fe406eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.290+0000 7f2fea251640 1 --2- 172.21.15.5:0/4134640437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe415aea0 0x7f2fe4141280 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.290+0000 7f2fea251640 1 -- 172.21.15.5:0/4134640437 >> 172.21.15.5:0/4134640437 conn(0x7f2fe40663d0 msgr2=0x7f2fe40667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.290+0000 7f2fea251640 1 -- 172.21.15.5:0/4134640437 shutdown_connections 2023-11-24T06:16:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.290+0000 7f2fea251640 1 -- 172.21.15.5:0/4134640437 wait complete. 2023-11-24T06:16:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.292+0000 7f2fea251640 1 Processor -- start 2023-11-24T06:16:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.292+0000 7f2fea251640 1 -- start start 2023-11-24T06:16:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.293+0000 7f2fea251640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe406f040 0x7f2fe4156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.293+0000 7f2fea251640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fe415aea0 0x7f2fe415d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.293+0000 7f2fea251640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fe415b280 0x7f2fe415ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.293+0000 7f2fea251640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4158130 con 0x7f2fe406f040 2023-11-24T06:16:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.293+0000 7f2fea251640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe415e400 con 0x7f2fe415b280 2023-11-24T06:16:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.293+0000 7f2fea251640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe4157ae0 con 0x7f2fe415aea0 2023-11-24T06:16:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.293+0000 7f2fe37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe406f040 0x7f2fe4156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.293+0000 7f2fe37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe406f040 0x7f2fe4156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41772/0 (socket says 172.21.15.5:41772) 2023-11-24T06:16:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.293+0000 7f2fe37fe640 1 -- 172.21.15.5:0/109513506 learned_addr learned my addr 172.21.15.5:0/109513506 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.294+0000 7f2fdbfff640 1 --2- 172.21.15.5:0/109513506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fe415aea0 0x7f2fe415d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.294+0000 7f2fe37fe640 1 -- 172.21.15.5:0/109513506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fe415aea0 msgr2=0x7f2fe415d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.294+0000 7f2fe37fe640 1 --2- 172.21.15.5:0/109513506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fe415aea0 0x7f2fe415d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.294+0000 7f2fe37fe640 1 -- 172.21.15.5:0/109513506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fe415b280 msgr2=0x7f2fe415ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.294+0000 7f2fe37fe640 1 --2- 172.21.15.5:0/109513506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fe415b280 0x7f2fe415ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.294+0000 7f2fe37fe640 1 -- 172.21.15.5:0/109513506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd40094d0 con 0x7f2fe406f040 2023-11-24T06:16:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.294+0000 7f2fe37fe640 1 --2- 172.21.15.5:0/109513506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe406f040 0x7f2fe4156770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2fd402b9e0 tx=0x7f2fd4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.294+0000 7f2fe0ff9640 1 -- 172.21.15.5:0/109513506 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4030850 con 0x7f2fe406f040 2023-11-24T06:16:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.294+0000 7f2fe0ff9640 1 -- 172.21.15.5:0/109513506 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd4043400 con 0x7f2fe406f040 2023-11-24T06:16:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.294+0000 7f2fe0ff9640 1 -- 172.21.15.5:0/109513506 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd40437b0 con 0x7f2fe406f040 2023-11-24T06:16:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.294+0000 7f2fea251640 1 -- 172.21.15.5:0/109513506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe4157dc0 con 0x7f2fe406f040 2023-11-24T06:16:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.295+0000 7f2fea251640 1 -- 172.21.15.5:0/109513506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe413eeb0 con 0x7f2fe406f040 2023-11-24T06:16:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.296+0000 7f2fea251640 1 -- 172.21.15.5:0/109513506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe4157920 con 0x7f2fe406f040 2023-11-24T06:16:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.297+0000 7f2fe0ff9640 1 -- 172.21.15.5:0/109513506 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2fd4043910 con 0x7f2fe406f040 2023-11-24T06:16:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.298+0000 7f2fe0ff9640 1 --2- 172.21.15.5:0/109513506 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2fc406ae40 0x7f2fc406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.298+0000 7f2fe0ff9640 1 -- 172.21.15.5:0/109513506 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fd403e070 con 0x7f2fe406f040 2023-11-24T06:16:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.298+0000 7f2fdbfff640 1 --2- 172.21.15.5:0/109513506 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2fc406ae40 0x7f2fc406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.299+0000 7f2fdbfff640 1 --2- 172.21.15.5:0/109513506 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2fc406ae40 0x7f2fc406d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2fe415b860 tx=0x7f2fcc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:10.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.302+0000 7f2fe0ff9640 1 -- 172.21.15.5:0/109513506 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2fd4077a70 con 0x7f2fe406f040 2023-11-24T06:16:10.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.535+0000 7f2fea251640 1 -- 172.21.15.5:0/109513506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2fe4074770 con 0x7f2fe406f040 2023-11-24T06:16:10.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.536+0000 7f2fe0ff9640 1 -- 172.21.15.5:0/109513506 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2fd4034070 con 0x7f2fe406f040 2023-11-24T06:16:10.540 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:16:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.539+0000 7f2fea251640 1 -- 172.21.15.5:0/109513506 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2fc406ae40 msgr2=0x7f2fc406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:10.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.539+0000 7f2fea251640 1 --2- 172.21.15.5:0/109513506 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2fc406ae40 0x7f2fc406d300 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2fe415b860 tx=0x7f2fcc008040 comp rx=0 tx=0).stop 2023-11-24T06:16:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.539+0000 7f2fea251640 1 -- 172.21.15.5:0/109513506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe406f040 msgr2=0x7f2fe4156770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.539+0000 7f2fea251640 1 --2- 172.21.15.5:0/109513506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe406f040 0x7f2fe4156770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2fd402b9e0 tx=0x7f2fd4004730 comp rx=0 tx=0).stop 2023-11-24T06:16:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.539+0000 7f2fea251640 1 -- 172.21.15.5:0/109513506 shutdown_connections 2023-11-24T06:16:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.539+0000 7f2fea251640 1 --2- 172.21.15.5:0/109513506 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2fc406ae40 0x7f2fc406d300 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.539+0000 7f2fea251640 1 --2- 172.21.15.5:0/109513506 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2fe415aea0 0x7f2fe415d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.539+0000 7f2fea251640 1 --2- 172.21.15.5:0/109513506 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2fe415b280 0x7f2fe415ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.539+0000 7f2fea251640 1 --2- 172.21.15.5:0/109513506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fe406f040 0x7f2fe4156770 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.539+0000 7f2fea251640 1 -- 172.21.15.5:0/109513506 >> 172.21.15.5:0/109513506 conn(0x7f2fe40663d0 msgr2=0x7f2fe413fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.539+0000 7f2fea251640 1 -- 172.21.15.5:0/109513506 shutdown_connections 2023-11-24T06:16:10.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:10.540+0000 7f2fea251640 1 -- 172.21.15.5:0/109513506 wait complete. 2023-11-24T06:16:10.563 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:16:10.563 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:16:10.564 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2023-11-24T06:16:10.564 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:16:10.564 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:16:10.564 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:16:10.564 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:16:10.564 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:16:10.564 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:16:10.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:10.565 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2023-11-24T06:16:10.620 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:16:10.620 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:16:10.620 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:16:10.620 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2023-11-24T06:16:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:10.621 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:16:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2023-11-24T06:16:10.621 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:16:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2023-11-24T06:16:10.621 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:16:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2023-11-24T06:16:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T06:16:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42321sec preferred_lft 42321sec 2023-11-24T06:16:10.622 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2023-11-24T06:16:10.622 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:10.622 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:16:10.622 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:10.622 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2023-11-24T06:16:10.622 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T06:16:10.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:10.623 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-11-24T06:16:10.623 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-11-24T06:16:10.623 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2023-11-24T06:16:10.623 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2023-11-24T06:16:10.623 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2023-11-24T06:16:10.623 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T06:16:10.623 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-11-24T06:16:11.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:11.089 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:16:11.175 INFO:teuthology.orchestra.run.smithi005.stdout:1 2023-11-24T06:16:11.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:11.179 DEBUG:teuthology.orchestra.run.smithi005:> route 2023-11-24T06:16:11.240 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2023-11-24T06:16:11.240 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:16:11.240 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T06:16:11.240 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T06:16:11.240 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:16:11.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:11.241 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-11-24T06:16:11.241 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-11-24T06:16:11.241 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T06:16:11.241 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T06:16:11.241 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T06:16:11.241 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-11-24T06:16:12.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:12.054 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2023-11-24T06:16:12.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:12.107 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2023-11-24T06:16:12.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:12.154 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-11-24T06:16:12.154 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-11-24T06:16:12.154 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:16:12.154 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-24T06:16:12.154 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-11-24T06:16:12.337 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-24T06:16:12.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:12.337 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-11-24T06:16:12.338 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-11-24T06:16:12.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-24T06:16:12.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T06:16:12.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-24T06:16:12.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-24T06:16:12.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-24T06:16:12.338 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-11-24T06:16:12.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:12.597 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-11-24T06:16:12.597 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-11-24T06:16:12.598 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2023-11-24T06:16:12.598 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T06:16:12.598 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-11-24T06:16:12.755 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T06:16:12.755 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:16:12.755 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:16:12.802 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:16:12.802 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:16:12.802 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:16:12.893 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2023-11-24T06:16:12.973 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:16:13.013 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2023-11-24T06:16:13.014 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2023-11-24T06:16:13.014 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2023-11-24T06:16:13.014 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2023-11-24T06:16:13.014 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2023-11-24T06:16:13.014 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2023-11-24T06:16:13.014 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2023-11-24T06:16:13.014 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2023-11-24T06:16:13.014 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2023-11-24T06:16:13.014 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2023-11-24T06:16:13.015 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2023-11-24T06:16:13.015 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:16:13.015 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2023-11-24T06:16:13.015 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2023-11-24T06:16:13.015 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2023-11-24T06:16:13.015 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2023-11-24T06:16:13.015 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2023-11-24T06:16:13.015 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2023-11-24T06:16:13.015 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:16:13.015 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:16:13.015 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:16:13.015 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:16:13.016 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2023-11-24T06:16:13.016 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2023-11-24T06:16:13.016 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:16:13.016 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:16:13.016 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:16:13.016 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:16:13.016 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2023-11-24T06:16:13.016 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:16:13.016 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:16:13.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:13.017 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2023-11-24T06:16:13.114 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:16:13.114 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T06:16:13.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2023-11-24T06:16:13.199 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:16:13.246 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2023-11-24T06:16:13.246 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2023-11-24T06:16:13.246 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2023-11-24T06:16:13.246 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2023-11-24T06:16:13.246 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2023-11-24T06:16:13.246 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2023-11-24T06:16:13.246 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2023-11-24T06:16:13.246 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2023-11-24T06:16:13.246 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2023-11-24T06:16:13.246 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2023-11-24T06:16:13.247 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2023-11-24T06:16:13.247 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:16:13.247 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2023-11-24T06:16:13.247 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2023-11-24T06:16:13.247 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2023-11-24T06:16:13.247 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2023-11-24T06:16:13.247 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2023-11-24T06:16:13.247 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2023-11-24T06:16:13.247 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:16:13.247 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:16:13.247 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:16:13.247 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:16:13.248 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2023-11-24T06:16:13.248 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2023-11-24T06:16:13.248 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:16:13.248 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:16:13.248 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:16:13.248 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:16:13.248 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2023-11-24T06:16:13.248 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:16:13.248 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:16:13.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:13.250 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2023-11-24T06:16:13.344 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:2023-11-24T06:16:13.341+0000 7f00b3a46480 -1 init, newargv = 0x5641d99c1830 newargc=15 2023-11-24T06:16:13.345 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[95799]: starting ceph client 2023-11-24T06:16:13.358 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[95799]: starting fuse 2023-11-24T06:16:14.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2023-11-24T06:16:14.507 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:16:14.571 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2023-11-24T06:16:14.571 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2023-11-24T06:16:14.571 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2023-11-24T06:16:14.571 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2023-11-24T06:16:14.571 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2023-11-24T06:16:14.571 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2023-11-24T06:16:14.572 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2023-11-24T06:16:14.572 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2023-11-24T06:16:14.572 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2023-11-24T06:16:14.572 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2023-11-24T06:16:14.572 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2023-11-24T06:16:14.572 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:16:14.572 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2023-11-24T06:16:14.572 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2023-11-24T06:16:14.572 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2023-11-24T06:16:14.572 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2023-11-24T06:16:14.572 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2023-11-24T06:16:14.572 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2023-11-24T06:16:14.573 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:16:14.573 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:16:14.573 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:16:14.573 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:16:14.573 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2023-11-24T06:16:14.573 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2023-11-24T06:16:14.573 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:16:14.573 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:16:14.573 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:16:14.573 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:16:14.573 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2023-11-24T06:16:14.573 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:16:14.573 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:16:14.574 INFO:teuthology.orchestra.run.smithi005.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:16:14.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:14.575 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2023-11-24T06:16:14.634 INFO:teuthology.orchestra.run.smithi005.stdout:40 2023-11-24T06:16:14.635 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-24T06:16:14.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:16:14.635 DEBUG:teuthology.orchestra.run.smithi005:> import glob 2023-11-24T06:16:14.635 DEBUG:teuthology.orchestra.run.smithi005:> import re 2023-11-24T06:16:14.635 DEBUG:teuthology.orchestra.run.smithi005:> import os 2023-11-24T06:16:14.635 DEBUG:teuthology.orchestra.run.smithi005:> import subprocess 2023-11-24T06:16:14.635 DEBUG:teuthology.orchestra.run.smithi005:> 2023-11-24T06:16:14.635 DEBUG:teuthology.orchestra.run.smithi005:> def _find_admin_socket(client_name): 2023-11-24T06:16:14.635 DEBUG:teuthology.orchestra.run.smithi005:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T06:16:14.635 DEBUG:teuthology.orchestra.run.smithi005:> files = glob.glob(asok_path) 2023-11-24T06:16:14.636 DEBUG:teuthology.orchestra.run.smithi005:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T06:16:14.636 DEBUG:teuthology.orchestra.run.smithi005:> 2023-11-24T06:16:14.636 DEBUG:teuthology.orchestra.run.smithi005:> # Given a non-glob path, it better be there 2023-11-24T06:16:14.636 DEBUG:teuthology.orchestra.run.smithi005:> if "*" not in asok_path: 2023-11-24T06:16:14.636 DEBUG:teuthology.orchestra.run.smithi005:> assert(len(files) == 1) 2023-11-24T06:16:14.636 DEBUG:teuthology.orchestra.run.smithi005:> return files[0] 2023-11-24T06:16:14.636 DEBUG:teuthology.orchestra.run.smithi005:> 2023-11-24T06:16:14.636 DEBUG:teuthology.orchestra.run.smithi005:> for f in files: 2023-11-24T06:16:14.636 DEBUG:teuthology.orchestra.run.smithi005:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:16:14.636 DEBUG:teuthology.orchestra.run.smithi005:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:16:14.636 DEBUG:teuthology.orchestra.run.smithi005:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:16:14.636 DEBUG:teuthology.orchestra.run.smithi005:> contents = proc_f.read() 2023-11-24T06:16:14.636 DEBUG:teuthology.orchestra.run.smithi005:> if mountpoint in contents: 2023-11-24T06:16:14.637 DEBUG:teuthology.orchestra.run.smithi005:> return f 2023-11-24T06:16:14.637 DEBUG:teuthology.orchestra.run.smithi005:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:16:14.637 DEBUG:teuthology.orchestra.run.smithi005:> 2023-11-24T06:16:14.637 DEBUG:teuthology.orchestra.run.smithi005:> print(_find_admin_socket("client.0")) 2023-11-24T06:16:14.637 DEBUG:teuthology.orchestra.run.smithi005:> ' 2023-11-24T06:16:14.828 INFO:teuthology.orchestra.run.smithi005.stdout:/var/run/ceph/ceph-client.0.95799.asok 2023-11-24T06:16:14.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:16:14.847 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.95799.asok 2023-11-24T06:16:14.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:14.847 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.95799.asok status 2023-11-24T06:16:15.096 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-11-24T06:16:15.097 INFO:teuthology.orchestra.run.smithi005.stdout: "metadata": { 2023-11-24T06:16:15.097 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:16:15.097 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:16:15.097 INFO:teuthology.orchestra.run.smithi005.stdout: "entity_id": "0", 2023-11-24T06:16:15.097 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": "smithi005", 2023-11-24T06:16:15.097 INFO:teuthology.orchestra.run.smithi005.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T06:16:15.097 INFO:teuthology.orchestra.run.smithi005.stdout: "pid": "95799", 2023-11-24T06:16:15.097 INFO:teuthology.orchestra.run.smithi005.stdout: "root": "/" 2023-11-24T06:16:15.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:16:15.097 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_count": 0, 2023-11-24T06:16:15.097 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_pinned_count": 0, 2023-11-24T06:16:15.097 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4656, 2023-11-24T06:16:15.098 INFO:teuthology.orchestra.run.smithi005.stdout: "inst": { 2023-11-24T06:16:15.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": { 2023-11-24T06:16:15.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "client", 2023-11-24T06:16:15.098 INFO:teuthology.orchestra.run.smithi005.stdout: "num": 4656 2023-11-24T06:16:15.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:16:15.098 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2023-11-24T06:16:15.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2023-11-24T06:16:15.098 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:16:15.098 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2398868598 2023-11-24T06:16:15.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:16:15.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:16:15.099 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2023-11-24T06:16:15.099 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2023-11-24T06:16:15.099 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:16:15.099 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2398868598 2023-11-24T06:16:15.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:16:15.100 INFO:teuthology.orchestra.run.smithi005.stdout: "inst_str": "client.4656 192.168.0.1:0/2398868598", 2023-11-24T06:16:15.100 INFO:teuthology.orchestra.run.smithi005.stdout: "addr_str": "192.168.0.1:0/2398868598", 2023-11-24T06:16:15.100 INFO:teuthology.orchestra.run.smithi005.stdout: "inode_count": 1, 2023-11-24T06:16:15.100 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_epoch": 8, 2023-11-24T06:16:15.100 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch": 22, 2023-11-24T06:16:15.100 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:16:15.100 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklisted": false, 2023-11-24T06:16:15.100 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs" 2023-11-24T06:16:15.101 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-11-24T06:16:15.112 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:16:15.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.331+0000 7fdf3b3ac640 1 Processor -- start 2023-11-24T06:16:15.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.332+0000 7fdf3b3ac640 1 -- start start 2023-11-24T06:16:15.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.333+0000 7fdf3b3ac640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdf340cf760 0x7fdf3415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:15.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.333+0000 7fdf3b3ac640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf34143950 0x7fdf3415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:15.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.333+0000 7fdf3b3ac640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf34071700 con 0x7fdf34143570 2023-11-24T06:16:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.334+0000 7fdf3b3ac640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3415af30 con 0x7fdf340cf760 2023-11-24T06:16:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.334+0000 7fdf3b3ac640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3415b410 con 0x7fdf34143950 2023-11-24T06:16:15.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.334+0000 7fdf39ba9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdf340cf760 0x7fdf3415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.334+0000 7fdf3abab640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf34143950 0x7fdf3415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.334+0000 7fdf39ba9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdf340cf760 0x7fdf3415a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:33258/0 (socket says 172.21.15.5:33258) 2023-11-24T06:16:15.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.334+0000 7fdf39ba9640 1 -- 172.21.15.5:0/3713365944 learned_addr learned my addr 172.21.15.5:0/3713365944 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.335+0000 7fdf39ba9640 1 -- 172.21.15.5:0/3713365944 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf34143950 msgr2=0x7fdf3415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:15.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.335+0000 7fdf39ba9640 1 --2- 172.21.15.5:0/3713365944 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf34143950 0x7fdf3415a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.335+0000 7fdf39ba9640 1 -- 172.21.15.5:0/3713365944 >> v1:172.21.15.6:6789/0 conn(0x7fdf34143570 legacy=0x7fdf340cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.335+0000 7fdf39ba9640 1 -- 172.21.15.5:0/3713365944 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3415b130 con 0x7fdf340cf760 2023-11-24T06:16:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.336+0000 7fdf39ba9640 1 --2- 172.21.15.5:0/3713365944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf340cf760 0x7fdf3415a4b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009870 tx=0x7fdf2802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0301a5ccb6bafc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.337+0000 7fdf393a8640 1 -- 172.21.15.5:0/3713365944 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28031640 con 0x7fdf340cf760 2023-11-24T06:16:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.337+0000 7fdf393a8640 1 -- 172.21.15.5:0/3713365944 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf280317a0 con 0x7fdf340cf760 2023-11-24T06:16:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.337+0000 7fdf393a8640 1 -- 172.21.15.5:0/3713365944 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28031b30 con 0x7fdf340cf760 2023-11-24T06:16:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.338+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/3713365944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf340cf760 msgr2=0x7fdf3415a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:15.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.338+0000 7fdf3b3ac640 1 --2- 172.21.15.5:0/3713365944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf340cf760 0x7fdf3415a4b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009870 tx=0x7fdf2802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:16:15.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.338+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/3713365944 shutdown_connections 2023-11-24T06:16:15.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.338+0000 7fdf3b3ac640 1 --2- 172.21.15.5:0/3713365944 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf34143950 0x7fdf3415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:15.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.338+0000 7fdf3b3ac640 1 --2- 172.21.15.5:0/3713365944 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf340cf760 0x7fdf3415a4b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:15.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.338+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/3713365944 >> 172.21.15.5:0/3713365944 conn(0x7fdf3406a860 msgr2=0x7fdf3406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:15.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.338+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/3713365944 shutdown_connections 2023-11-24T06:16:15.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.338+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/3713365944 wait complete. 2023-11-24T06:16:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.340+0000 7fdf3b3ac640 1 Processor -- start 2023-11-24T06:16:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.340+0000 7fdf3b3ac640 1 -- start start 2023-11-24T06:16:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.341+0000 7fdf3b3ac640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf340cf760 0x7fdf3413a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.341+0000 7fdf3b3ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34143570 0x7fdf340ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:15.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.341+0000 7fdf3b3ac640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf34143950 0x7fdf3413bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.341+0000 7fdf3b3ac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf340cdf80 con 0x7fdf34143570 2023-11-24T06:16:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.341+0000 7fdf3b3ac640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf340cd650 con 0x7fdf340cf760 2023-11-24T06:16:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.341+0000 7fdf3b3ac640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf340cd930 con 0x7fdf34143950 2023-11-24T06:16:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.341+0000 7fdf39ba9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34143570 0x7fdf340ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.342+0000 7fdf39ba9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34143570 0x7fdf340ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50880/0 (socket says 172.21.15.5:50880) 2023-11-24T06:16:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.342+0000 7fdf39ba9640 1 -- 172.21.15.5:0/1904839834 learned_addr learned my addr 172.21.15.5:0/1904839834 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.342+0000 7fdf3a3aa640 1 --2- 172.21.15.5:0/1904839834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf340cf760 0x7fdf3413a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.342+0000 7fdf3abab640 1 --2- 172.21.15.5:0/1904839834 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf34143950 0x7fdf3413bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.342+0000 7fdf39ba9640 1 -- 172.21.15.5:0/1904839834 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf34143950 msgr2=0x7fdf3413bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.342+0000 7fdf39ba9640 1 --2- 172.21.15.5:0/1904839834 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf34143950 0x7fdf3413bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:15.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.342+0000 7fdf39ba9640 1 -- 172.21.15.5:0/1904839834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf340cf760 msgr2=0x7fdf3413a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.342+0000 7fdf39ba9640 1 --2- 172.21.15.5:0/1904839834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf340cf760 0x7fdf3413a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:15.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.342+0000 7fdf39ba9640 1 -- 172.21.15.5:0/1904839834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf280094d0 con 0x7fdf34143570 2023-11-24T06:16:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.342+0000 7fdf39ba9640 1 --2- 172.21.15.5:0/1904839834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34143570 0x7fdf340ce410 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdf28000c00 tx=0x7fdf280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.343+0000 7fdf32ffd640 1 -- 172.21.15.5:0/1904839834 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2803a930 con 0x7fdf34143570 2023-11-24T06:16:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.343+0000 7fdf32ffd640 1 -- 172.21.15.5:0/1904839834 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2803aa90 con 0x7fdf34143570 2023-11-24T06:16:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.343+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/1904839834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf340cdc10 con 0x7fdf34143570 2023-11-24T06:16:15.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.343+0000 7fdf32ffd640 1 -- 172.21.15.5:0/1904839834 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2803ae40 con 0x7fdf34143570 2023-11-24T06:16:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.343+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/1904839834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3413c4a0 con 0x7fdf34143570 2023-11-24T06:16:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.344+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/1904839834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf340cdda0 con 0x7fdf34143570 2023-11-24T06:16:15.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.345+0000 7fdf32ffd640 1 -- 172.21.15.5:0/1904839834 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdf2802f050 con 0x7fdf34143570 2023-11-24T06:16:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.346+0000 7fdf32ffd640 1 --2- 172.21.15.5:0/1904839834 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf2006ae70 0x7fdf2006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.346+0000 7fdf32ffd640 1 -- 172.21.15.5:0/1904839834 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf280ab450 con 0x7fdf34143570 2023-11-24T06:16:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.346+0000 7fdf3a3aa640 1 --2- 172.21.15.5:0/1904839834 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf2006ae70 0x7fdf2006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:15.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.347+0000 7fdf3a3aa640 1 --2- 172.21.15.5:0/1904839834 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf2006ae70 0x7fdf2006d330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf341392e0 tx=0x7fdf24009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:15.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.351+0000 7fdf32ffd640 1 -- 172.21.15.5:0/1904839834 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf280764f0 con 0x7fdf34143570 2023-11-24T06:16:15.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.583+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/1904839834 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdf3404e840 con 0x7fdf34143570 2023-11-24T06:16:15.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.584+0000 7fdf32ffd640 1 -- 172.21.15.5:0/1904839834 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdf28079b10 con 0x7fdf34143570 2023-11-24T06:16:15.586 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:16:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.586+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/1904839834 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf2006ae70 msgr2=0x7fdf2006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.586+0000 7fdf3b3ac640 1 --2- 172.21.15.5:0/1904839834 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf2006ae70 0x7fdf2006d330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdf341392e0 tx=0x7fdf24009040 comp rx=0 tx=0).stop 2023-11-24T06:16:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.586+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/1904839834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34143570 msgr2=0x7fdf340ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.586+0000 7fdf3b3ac640 1 --2- 172.21.15.5:0/1904839834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34143570 0x7fdf340ce410 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdf28000c00 tx=0x7fdf280046c0 comp rx=0 tx=0).stop 2023-11-24T06:16:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.587+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/1904839834 shutdown_connections 2023-11-24T06:16:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.587+0000 7fdf3b3ac640 1 --2- 172.21.15.5:0/1904839834 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf2006ae70 0x7fdf2006d330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.587+0000 7fdf3b3ac640 1 --2- 172.21.15.5:0/1904839834 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf34143950 0x7fdf3413bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:15.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.587+0000 7fdf3b3ac640 1 --2- 172.21.15.5:0/1904839834 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf34143570 0x7fdf340ce410 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.587+0000 7fdf3b3ac640 1 --2- 172.21.15.5:0/1904839834 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf340cf760 0x7fdf3413a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.587+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/1904839834 >> 172.21.15.5:0/1904839834 conn(0x7fdf3406a860 msgr2=0x7fdf3406ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.587+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/1904839834 shutdown_connections 2023-11-24T06:16:15.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:15.587+0000 7fdf3b3ac640 1 -- 172.21.15.5:0/1904839834 wait complete. 2023-11-24T06:16:15.607 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:16:15.607 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:16:15.607 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2023-11-24T06:16:15.607 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-24T06:16:15.608 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T06:16:15.608 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:16:15.608 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:16:15.608 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:16:15.608 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-24T06:16:15.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:15.608 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2023-11-24T06:16:15.667 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:16:15.668 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:16:15.668 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:16:15.668 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:15.668 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2023-11-24T06:16:15.668 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:15.668 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:16:15.668 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:15.668 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2023-11-24T06:16:15.668 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:16:15.669 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:15.669 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2023-11-24T06:16:15.669 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:16:15.669 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:15.669 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2023-11-24T06:16:15.669 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T06:16:15.669 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42316sec preferred_lft 42316sec 2023-11-24T06:16:15.669 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2023-11-24T06:16:15.669 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:15.669 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:16:15.669 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:15.669 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2023-11-24T06:16:15.670 INFO:teuthology.orchestra.run.smithi005.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:16:15.670 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether f6:90:1f:88:d8:74 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:15.670 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:16:15.670 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:15.670 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::a4c7:50ff:feb3:591d/64 scope link 2023-11-24T06:16:15.670 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:15.670 INFO:teuthology.orchestra.run.smithi005.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:16:15.671 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether f6:90:1f:88:d8:74 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:16:15.671 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::f490:1fff:fe88:d874/64 scope link 2023-11-24T06:16:15.671 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:15.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:15.672 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:16:15.770 INFO:teuthology.orchestra.run.smithi005.stdout:1 2023-11-24T06:16:15.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:15.775 DEBUG:teuthology.orchestra.run.smithi005:> route 2023-11-24T06:16:15.837 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2023-11-24T06:16:15.837 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:16:15.838 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T06:16:15.838 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T06:16:15.838 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:16:15.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:15.838 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-11-24T06:16:15.839 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-11-24T06:16:15.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T06:16:15.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T06:16:15.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T06:16:15.839 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-11-24T06:16:16.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:16.038 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2023-11-24T06:16:16.094 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:16:16.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:16.096 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2023-11-24T06:16:16.152 INFO:teuthology.orchestra.run.smithi005.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-24T06:16:16.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:16.153 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-11-24T06:16:16.153 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-11-24T06:16:16.153 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:16:16.154 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-24T06:16:16.154 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-11-24T06:16:16.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:16.311 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T06:16:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:16:16.412 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:16:16.412 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:16:16.412 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:16.412 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2023-11-24T06:16:16.412 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:16:16.412 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0a:64:f4:0a:4b:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:16:16.412 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:16:16.412 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:16.413 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::864:f4ff:fe0a:4b01/64 scope link 2023-11-24T06:16:16.413 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:16.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:16.416 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T06:16:16.509 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:16:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:16:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:16:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:16.509 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2023-11-24T06:16:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:16.510 INFO:teuthology.orchestra.run.smithi005.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:16:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0a:64:f4:0a:4b:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:16:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:16:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::864:f4ff:fe0a:4b01/64 scope link 2023-11-24T06:16:16.510 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:16.513 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-24T06:16:16.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:16.514 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-11-24T06:16:16.514 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-11-24T06:16:16.514 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-24T06:16:16.514 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T06:16:16.515 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-24T06:16:16.515 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-24T06:16:16.515 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-24T06:16:16.515 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-11-24T06:16:16.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:16.783 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-11-24T06:16:16.783 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-11-24T06:16:16.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.1 up 2023-11-24T06:16:16.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T06:16:16.784 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-11-24T06:16:16.953 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-24T06:16:16.954 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:16:16.954 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-24T06:16:17.006 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-24T06:16:17.007 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:16:17.007 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-24T06:16:17.061 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2023-11-24T06:16:17.164 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:16:17.223 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2023-11-24T06:16:17.223 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2023-11-24T06:16:17.223 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2023-11-24T06:16:17.223 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2023-11-24T06:16:17.224 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2023-11-24T06:16:17.224 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2023-11-24T06:16:17.224 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2023-11-24T06:16:17.224 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2023-11-24T06:16:17.224 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2023-11-24T06:16:17.224 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2023-11-24T06:16:17.224 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2023-11-24T06:16:17.224 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:16:17.224 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2023-11-24T06:16:17.224 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2023-11-24T06:16:17.224 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2023-11-24T06:16:17.224 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2023-11-24T06:16:17.225 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2023-11-24T06:16:17.225 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2023-11-24T06:16:17.225 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:16:17.225 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:16:17.225 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:16:17.225 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:16:17.225 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2023-11-24T06:16:17.225 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2023-11-24T06:16:17.225 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:16:17.225 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:16:17.225 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:16:17.225 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:16:17.226 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2023-11-24T06:16:17.226 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:16:17.226 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:16:17.226 INFO:teuthology.orchestra.run.smithi005.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:16:17.226 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:16:17.226 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:16:17.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:17.227 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2023-11-24T06:16:17.282 INFO:teuthology.orchestra.run.smithi005.stdout:40 2023-11-24T06:16:17.283 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2023-11-24T06:16:17.283 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-11-24T06:16:17.289 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2023-11-24T06:16:17.371 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:16:17.416 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2023-11-24T06:16:17.416 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2023-11-24T06:16:17.416 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2023-11-24T06:16:17.416 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2023-11-24T06:16:17.416 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2023-11-24T06:16:17.416 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2023-11-24T06:16:17.416 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2023-11-24T06:16:17.416 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2023-11-24T06:16:17.416 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2023-11-24T06:16:17.416 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2023-11-24T06:16:17.416 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2023-11-24T06:16:17.416 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:16:17.417 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2023-11-24T06:16:17.417 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2023-11-24T06:16:17.417 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2023-11-24T06:16:17.417 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2023-11-24T06:16:17.417 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2023-11-24T06:16:17.417 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2023-11-24T06:16:17.417 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:16:17.417 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:16:17.417 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:16:17.417 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:16:17.417 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2023-11-24T06:16:17.418 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2023-11-24T06:16:17.418 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:16:17.418 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:16:17.418 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:16:17.418 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:16:17.418 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2023-11-24T06:16:17.418 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:16:17.418 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:16:17.418 INFO:teuthology.orchestra.run.smithi005.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-24T06:16:17.418 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:16:17.418 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:16:17.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:17.419 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2023-11-24T06:16:17.486 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi005.stderr:2023-11-24T06:16:17.484+0000 7fede3244480 -1 init, newargv = 0x559715d3bb70 newargc=15 2023-11-24T06:16:17.486 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi005.stderr:ceph-fuse[96345]: starting ceph client 2023-11-24T06:16:17.497 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi005.stderr:ceph-fuse[96345]: starting fuse 2023-11-24T06:16:17.505 INFO:teuthology.orchestra.run.smithi005.stdout:40 2023-11-24T06:16:17.506 INFO:teuthology.orchestra.run.smithi005.stdout:42 2023-11-24T06:16:17.506 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 42] 2023-11-24T06:16:17.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:16:17.506 DEBUG:teuthology.orchestra.run.smithi005:> import glob 2023-11-24T06:16:17.507 DEBUG:teuthology.orchestra.run.smithi005:> import re 2023-11-24T06:16:17.507 DEBUG:teuthology.orchestra.run.smithi005:> import os 2023-11-24T06:16:17.507 DEBUG:teuthology.orchestra.run.smithi005:> import subprocess 2023-11-24T06:16:17.507 DEBUG:teuthology.orchestra.run.smithi005:> 2023-11-24T06:16:17.507 DEBUG:teuthology.orchestra.run.smithi005:> def _find_admin_socket(client_name): 2023-11-24T06:16:17.507 DEBUG:teuthology.orchestra.run.smithi005:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-24T06:16:17.507 DEBUG:teuthology.orchestra.run.smithi005:> files = glob.glob(asok_path) 2023-11-24T06:16:17.507 DEBUG:teuthology.orchestra.run.smithi005:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-24T06:16:17.507 DEBUG:teuthology.orchestra.run.smithi005:> 2023-11-24T06:16:17.508 DEBUG:teuthology.orchestra.run.smithi005:> # Given a non-glob path, it better be there 2023-11-24T06:16:17.508 DEBUG:teuthology.orchestra.run.smithi005:> if "*" not in asok_path: 2023-11-24T06:16:17.508 DEBUG:teuthology.orchestra.run.smithi005:> assert(len(files) == 1) 2023-11-24T06:16:17.508 DEBUG:teuthology.orchestra.run.smithi005:> return files[0] 2023-11-24T06:16:17.508 DEBUG:teuthology.orchestra.run.smithi005:> 2023-11-24T06:16:17.508 DEBUG:teuthology.orchestra.run.smithi005:> for f in files: 2023-11-24T06:16:17.508 DEBUG:teuthology.orchestra.run.smithi005:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:16:17.508 DEBUG:teuthology.orchestra.run.smithi005:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:16:17.508 DEBUG:teuthology.orchestra.run.smithi005:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:16:17.508 DEBUG:teuthology.orchestra.run.smithi005:> contents = proc_f.read() 2023-11-24T06:16:17.508 DEBUG:teuthology.orchestra.run.smithi005:> if mountpoint in contents: 2023-11-24T06:16:17.509 DEBUG:teuthology.orchestra.run.smithi005:> return f 2023-11-24T06:16:17.509 DEBUG:teuthology.orchestra.run.smithi005:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:16:17.509 DEBUG:teuthology.orchestra.run.smithi005:> 2023-11-24T06:16:17.509 DEBUG:teuthology.orchestra.run.smithi005:> print(_find_admin_socket("client.2")) 2023-11-24T06:16:17.509 DEBUG:teuthology.orchestra.run.smithi005:> ' 2023-11-24T06:16:17.717 INFO:teuthology.orchestra.run.smithi005.stdout:/var/run/ceph/ceph-client.2.96345.asok 2023-11-24T06:16:17.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:16:17.735 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.96345.asok 2023-11-24T06:16:17.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:17.735 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.96345.asok status 2023-11-24T06:16:17.989 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-11-24T06:16:17.989 INFO:teuthology.orchestra.run.smithi005.stdout: "metadata": { 2023-11-24T06:16:17.989 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:16:17.989 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:16:17.989 INFO:teuthology.orchestra.run.smithi005.stdout: "entity_id": "2", 2023-11-24T06:16:17.989 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": "smithi005", 2023-11-24T06:16:17.990 INFO:teuthology.orchestra.run.smithi005.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-24T06:16:17.990 INFO:teuthology.orchestra.run.smithi005.stdout: "pid": "96345", 2023-11-24T06:16:17.990 INFO:teuthology.orchestra.run.smithi005.stdout: "root": "/" 2023-11-24T06:16:17.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:16:17.990 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_count": 0, 2023-11-24T06:16:17.990 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_pinned_count": 0, 2023-11-24T06:16:17.990 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4665, 2023-11-24T06:16:17.990 INFO:teuthology.orchestra.run.smithi005.stdout: "inst": { 2023-11-24T06:16:17.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": { 2023-11-24T06:16:17.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "client", 2023-11-24T06:16:17.990 INFO:teuthology.orchestra.run.smithi005.stdout: "num": 4665 2023-11-24T06:16:17.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:16:17.991 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2023-11-24T06:16:17.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2023-11-24T06:16:17.991 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:16:17.991 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 729705551 2023-11-24T06:16:17.991 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:16:17.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:16:17.991 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2023-11-24T06:16:17.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2023-11-24T06:16:17.991 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:16:17.991 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 729705551 2023-11-24T06:16:17.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:16:17.991 INFO:teuthology.orchestra.run.smithi005.stdout: "inst_str": "client.4665 192.168.0.2:0/729705551", 2023-11-24T06:16:17.992 INFO:teuthology.orchestra.run.smithi005.stdout: "addr_str": "192.168.0.2:0/729705551", 2023-11-24T06:16:17.992 INFO:teuthology.orchestra.run.smithi005.stdout: "inode_count": 1, 2023-11-24T06:16:17.992 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_epoch": 8, 2023-11-24T06:16:17.992 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch": 22, 2023-11-24T06:16:17.992 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:16:17.992 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklisted": false, 2023-11-24T06:16:17.992 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs" 2023-11-24T06:16:17.992 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-11-24T06:16:18.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:16:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.224+0000 7fb6c7e7c640 1 Processor -- start 2023-11-24T06:16:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.225+0000 7fb6c7e7c640 1 -- start start 2023-11-24T06:16:18.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.226+0000 7fb6c7e7c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb6c0145f80 0x7fb6c00b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:18.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.226+0000 7fb6c7e7c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6c006f1a0 0x7fb6c006bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:18.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.226+0000 7fb6c7e7c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c015d970 con 0x7fb6c00d0fe0 2023-11-24T06:16:18.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.226+0000 7fb6c7e7c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c00b4f80 con 0x7fb6c0145f80 2023-11-24T06:16:18.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.226+0000 7fb6c7e7c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00d14e0 con 0x7fb6c006f1a0 2023-11-24T06:16:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.227+0000 7fb6c5c18640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb6c0145f80 0x7fb6c00b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.227+0000 7fb6c5417640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb6c00d0fe0 0x7fb6c006ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35642/0 (socket says 172.21.15.5:35642) 2023-11-24T06:16:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.227+0000 7fb6c5417640 1 -- 172.21.15.5:0/3294842089 learned_addr learned my addr 172.21.15.5:0/3294842089 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.227+0000 7fb6c6419640 1 --2- 172.21.15.5:0/3294842089 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6c006f1a0 0x7fb6c006bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.227+0000 7fb6c5c18640 1 -- 172.21.15.5:0/3294842089 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6c006f1a0 msgr2=0x7fb6c006bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.227+0000 7fb6c5c18640 1 --2- 172.21.15.5:0/3294842089 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6c006f1a0 0x7fb6c006bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:18.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.227+0000 7fb6c5c18640 1 -- 172.21.15.5:0/3294842089 >> v1:172.21.15.5:6789/0 conn(0x7fb6c00d0fe0 legacy=0x7fb6c006ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:18.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.227+0000 7fb6c5c18640 1 -- 172.21.15.5:0/3294842089 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c00d17c0 con 0x7fb6c0145f80 2023-11-24T06:16:18.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.227+0000 7fb6c4c16640 1 -- 172.21.15.5:0/3294842089 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897488909 0 0) 0x7fb6b0001110 con 0x7fb6c00d0fe0 2023-11-24T06:16:18.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.228+0000 7fb6c5c18640 1 --2- 172.21.15.5:0/3294842089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c0145f80 0x7fb6c00b4a40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0097a0 tx=0x7fb6bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417472dc02711155 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:18.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.229+0000 7fb6c4c16640 1 -- 172.21.15.5:0/3294842089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc031830 con 0x7fb6c0145f80 2023-11-24T06:16:18.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.229+0000 7fb6c4c16640 1 -- 172.21.15.5:0/3294842089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6bc031990 con 0x7fb6c0145f80 2023-11-24T06:16:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.229+0000 7fb6c4c16640 1 -- 172.21.15.5:0/3294842089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc031d20 con 0x7fb6c0145f80 2023-11-24T06:16:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.229+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/3294842089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c0145f80 msgr2=0x7fb6c00b4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:18.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.229+0000 7fb6c7e7c640 1 --2- 172.21.15.5:0/3294842089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c0145f80 0x7fb6c00b4a40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0097a0 tx=0x7fb6bc02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:16:18.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.229+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/3294842089 shutdown_connections 2023-11-24T06:16:18.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.229+0000 7fb6c7e7c640 1 --2- 172.21.15.5:0/3294842089 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6c006f1a0 0x7fb6c006bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:18.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.229+0000 7fb6c7e7c640 1 --2- 172.21.15.5:0/3294842089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c0145f80 0x7fb6c00b4a40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:18.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.229+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/3294842089 >> 172.21.15.5:0/3294842089 conn(0x7fb6c00663d0 msgr2=0x7fb6c00667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:18.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.230+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/3294842089 shutdown_connections 2023-11-24T06:16:18.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.230+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/3294842089 wait complete. 2023-11-24T06:16:18.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.231+0000 7fb6c7e7c640 1 Processor -- start 2023-11-24T06:16:18.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.231+0000 7fb6c7e7c640 1 -- start start 2023-11-24T06:16:18.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.232+0000 7fb6c7e7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c006f1a0 0x7fb6c01f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:18.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.232+0000 7fb6c7e7c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6c00d0fe0 0x7fb6c01f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:18.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.232+0000 7fb6c7e7c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c0145f80 0x7fb6c01f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:18.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.232+0000 7fb6c7e7c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c014fff0 con 0x7fb6c006f1a0 2023-11-24T06:16:18.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.232+0000 7fb6c7e7c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c014f3e0 con 0x7fb6c0145f80 2023-11-24T06:16:18.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.232+0000 7fb6c7e7c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c014f6c0 con 0x7fb6c00d0fe0 2023-11-24T06:16:18.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.233+0000 7fb6c5c18640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c006f1a0 0x7fb6c01f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:18.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.233+0000 7fb6c5c18640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c006f1a0 0x7fb6c01f2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50906/0 (socket says 172.21.15.5:50906) 2023-11-24T06:16:18.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.233+0000 7fb6c5c18640 1 -- 172.21.15.5:0/1470055899 learned_addr learned my addr 172.21.15.5:0/1470055899 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:18.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.233+0000 7fb6c5417640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6c00d0fe0 0x7fb6c01f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:18.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.233+0000 7fb6c6419640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c0145f80 0x7fb6c01f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:18.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.233+0000 7fb6c5c18640 1 -- 172.21.15.5:0/1470055899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6c00d0fe0 msgr2=0x7fb6c01f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:18.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.233+0000 7fb6c5c18640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6c00d0fe0 0x7fb6c01f6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:18.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.233+0000 7fb6c5c18640 1 -- 172.21.15.5:0/1470055899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c0145f80 msgr2=0x7fb6c01f68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:18.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.233+0000 7fb6c5c18640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c0145f80 0x7fb6c01f68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:18.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.233+0000 7fb6c5c18640 1 -- 172.21.15.5:0/1470055899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc009400 con 0x7fb6c006f1a0 2023-11-24T06:16:18.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.233+0000 7fb6c5417640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6c00d0fe0 0x7fb6c01f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:18.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.234+0000 7fb6c5c18640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c006f1a0 0x7fb6c01f2870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc000c00 tx=0x7fb6bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:18.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.234+0000 7fb6b67fc640 1 -- 172.21.15.5:0/1470055899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc03aa60 con 0x7fb6c006f1a0 2023-11-24T06:16:18.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.234+0000 7fb6b67fc640 1 -- 172.21.15.5:0/1470055899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6bc03abc0 con 0x7fb6c006f1a0 2023-11-24T06:16:18.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.234+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/1470055899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c014f9a0 con 0x7fb6c006f1a0 2023-11-24T06:16:18.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.234+0000 7fb6b67fc640 1 -- 172.21.15.5:0/1470055899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0443f0 con 0x7fb6c006f1a0 2023-11-24T06:16:18.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.234+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/1470055899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c0143cd0 con 0x7fb6c006f1a0 2023-11-24T06:16:18.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.235+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/1470055899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c0074770 con 0x7fb6c006f1a0 2023-11-24T06:16:18.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.236+0000 7fb6b67fc640 1 -- 172.21.15.5:0/1470055899 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb6bc04d450 con 0x7fb6c006f1a0 2023-11-24T06:16:18.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.237+0000 7fb6b67fc640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb6a406ae40 0x7fb6a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:18.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.237+0000 7fb6b67fc640 1 -- 172.21.15.5:0/1470055899 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6bc0ab740 con 0x7fb6c006f1a0 2023-11-24T06:16:18.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.238+0000 7fb6c5417640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb6a406ae40 0x7fb6a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:18.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.238+0000 7fb6c5417640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb6a406ae40 0x7fb6a406d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0145990 tx=0x7fb6b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:18.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.242+0000 7fb6b67fc640 1 -- 172.21.15.5:0/1470055899 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6bc0767f0 con 0x7fb6c006f1a0 2023-11-24T06:16:18.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.479+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/1470055899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb6c0070de0 con 0x7fb6c006f1a0 2023-11-24T06:16:18.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.480+0000 7fb6b67fc640 1 -- 172.21.15.5:0/1470055899 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb6bc035070 con 0x7fb6c006f1a0 2023-11-24T06:16:18.482 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:16:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.483+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/1470055899 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb6a406ae40 msgr2=0x7fb6a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.483+0000 7fb6c7e7c640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb6a406ae40 0x7fb6a406d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0145990 tx=0x7fb6b0009040 comp rx=0 tx=0).stop 2023-11-24T06:16:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.483+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/1470055899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c006f1a0 msgr2=0x7fb6c01f2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:18.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.483+0000 7fb6c7e7c640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c006f1a0 0x7fb6c01f2870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc000c00 tx=0x7fb6bc0046c0 comp rx=0 tx=0).stop 2023-11-24T06:16:18.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.483+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/1470055899 shutdown_connections 2023-11-24T06:16:18.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.483+0000 7fb6c7e7c640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb6a406ae40 0x7fb6a406d300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:18.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.483+0000 7fb6c7e7c640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb6c00d0fe0 0x7fb6c01f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:18.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.483+0000 7fb6c7e7c640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb6c0145f80 0x7fb6c01f68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:18.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.483+0000 7fb6c7e7c640 1 --2- 172.21.15.5:0/1470055899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6c006f1a0 0x7fb6c01f2870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:18.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.483+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/1470055899 >> 172.21.15.5:0/1470055899 conn(0x7fb6c00663d0 msgr2=0x7fb6c006bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:18.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.483+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/1470055899 shutdown_connections 2023-11-24T06:16:18.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:18.483+0000 7fb6c7e7c640 1 -- 172.21.15.5:0/1470055899 wait complete. 2023-11-24T06:16:18.503 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:16:18.503 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:16:18.503 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2023-11-24T06:16:18.503 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:16:18.503 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:16:18.503 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:16:18.504 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:16:18.504 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:16:18.504 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:16:18.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:18.504 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2023-11-24T06:16:18.568 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:16:18.568 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:16:18.568 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:16:18.568 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:18.569 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-24T06:16:18.569 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:18.569 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:16:18.569 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:18.569 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f0 2023-11-24T06:16:18.569 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:16:18.569 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:18.569 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f1 2023-11-24T06:16:18.569 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:16:18.569 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:18.570 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f0 2023-11-24T06:16:18.570 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T06:16:18.570 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 42321sec preferred_lft 42321sec 2023-11-24T06:16:18.570 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link noprefixroute 2023-11-24T06:16:18.570 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:18.570 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:16:18.570 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:18.570 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f1 2023-11-24T06:16:18.571 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T06:16:18.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:18.572 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-24T06:16:18.572 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-24T06:16:18.572 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add name ceph-brx type bridge 2023-11-24T06:16:18.572 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr flush dev ceph-brx 2023-11-24T06:16:18.572 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set ceph-brx up 2023-11-24T06:16:18.572 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T06:16:18.572 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-24T06:16:19.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:19.146 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:16:19.235 INFO:teuthology.orchestra.run.smithi006.stdout:1 2023-11-24T06:16:19.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:19.239 DEBUG:teuthology.orchestra.run.smithi006:> route 2023-11-24T06:16:19.287 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2023-11-24T06:16:19.287 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:16:19.287 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T06:16:19.287 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T06:16:19.287 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:16:19.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:19.288 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-24T06:16:19.288 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-24T06:16:19.288 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T06:16:19.288 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T06:16:19.288 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T06:16:19.288 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-24T06:16:20.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:20.084 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-24T06:16:20.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:20.134 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list-id 2023-11-24T06:16:20.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:20.179 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-24T06:16:20.179 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-24T06:16:20.179 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:16:20.179 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-24T06:16:20.179 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-24T06:16:20.366 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-24T06:16:20.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:20.366 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-24T06:16:20.367 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-24T06:16:20.367 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-24T06:16:20.367 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T06:16:20.367 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-24T06:16:20.367 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-24T06:16:20.367 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-24T06:16:20.367 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-24T06:16:20.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:20.631 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-24T06:16:20.631 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-24T06:16:20.631 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2023-11-24T06:16:20.631 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T06:16:20.631 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-24T06:16:20.787 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T06:16:20.787 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:16:20.787 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:16:20.836 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:16:20.836 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:16:20.837 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:16:20.886 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-24T06:16:20.982 DEBUG:teuthology.orchestra.run.smithi006:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:16:21.040 INFO:teuthology.orchestra.run.smithi006.stdout:/proc 2023-11-24T06:16:21.040 INFO:teuthology.orchestra.run.smithi006.stdout:/sys 2023-11-24T06:16:21.041 INFO:teuthology.orchestra.run.smithi006.stdout:/dev 2023-11-24T06:16:21.041 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/security 2023-11-24T06:16:21.041 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/shm 2023-11-24T06:16:21.041 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/pts 2023-11-24T06:16:21.041 INFO:teuthology.orchestra.run.smithi006.stdout:/run 2023-11-24T06:16:21.041 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/cgroup 2023-11-24T06:16:21.041 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/pstore 2023-11-24T06:16:21.041 INFO:teuthology.orchestra.run.smithi006.stdout:/ 2023-11-24T06:16:21.041 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/selinux 2023-11-24T06:16:21.042 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:16:21.042 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/hugepages 2023-11-24T06:16:21.042 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/mqueue 2023-11-24T06:16:21.042 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/debug 2023-11-24T06:16:21.042 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/tracing 2023-11-24T06:16:21.042 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/fuse/connections 2023-11-24T06:16:21.042 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/config 2023-11-24T06:16:21.042 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:16:21.042 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:16:21.042 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:16:21.042 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:16:21.042 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/1000 2023-11-24T06:16:21.043 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph 2023-11-24T06:16:21.043 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:16:21.043 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:16:21.043 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:16:21.043 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:16:21.043 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns 2023-11-24T06:16:21.043 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:16:21.043 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:16:21.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:21.044 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-24T06:16:21.097 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:16:21.098 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T06:16:21.104 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-24T06:16:21.179 DEBUG:teuthology.orchestra.run.smithi006:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:16:21.220 INFO:teuthology.orchestra.run.smithi006.stdout:/proc 2023-11-24T06:16:21.221 INFO:teuthology.orchestra.run.smithi006.stdout:/sys 2023-11-24T06:16:21.221 INFO:teuthology.orchestra.run.smithi006.stdout:/dev 2023-11-24T06:16:21.221 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/security 2023-11-24T06:16:21.221 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/shm 2023-11-24T06:16:21.221 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/pts 2023-11-24T06:16:21.221 INFO:teuthology.orchestra.run.smithi006.stdout:/run 2023-11-24T06:16:21.222 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/cgroup 2023-11-24T06:16:21.222 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/pstore 2023-11-24T06:16:21.222 INFO:teuthology.orchestra.run.smithi006.stdout:/ 2023-11-24T06:16:21.222 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/selinux 2023-11-24T06:16:21.222 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:16:21.222 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/hugepages 2023-11-24T06:16:21.222 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/mqueue 2023-11-24T06:16:21.222 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/debug 2023-11-24T06:16:21.222 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/tracing 2023-11-24T06:16:21.223 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/fuse/connections 2023-11-24T06:16:21.223 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/config 2023-11-24T06:16:21.223 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:16:21.223 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:16:21.223 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:16:21.223 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:16:21.223 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/1000 2023-11-24T06:16:21.223 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph 2023-11-24T06:16:21.224 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:16:21.224 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:16:21.224 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:16:21.224 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:16:21.224 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns 2023-11-24T06:16:21.224 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:16:21.224 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:16:21.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:21.226 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-24T06:16:21.302 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi006.stderr:2023-11-24T06:16:21.299+0000 7f2cdbe36480 -1 init, newargv = 0x55da8b8d2fd0 newargc=15 2023-11-24T06:16:21.302 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi006.stderr:ceph-fuse[93466]: starting ceph client 2023-11-24T06:16:21.313 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi006.stderr:ceph-fuse[93466]: starting fuse 2023-11-24T06:16:21.317 INFO:teuthology.orchestra.run.smithi006.stdout:41 2023-11-24T06:16:21.317 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T06:16:21.317 DEBUG:teuthology.orchestra.run.smithi006:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:16:21.318 DEBUG:teuthology.orchestra.run.smithi006:> import glob 2023-11-24T06:16:21.318 DEBUG:teuthology.orchestra.run.smithi006:> import re 2023-11-24T06:16:21.318 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-24T06:16:21.318 DEBUG:teuthology.orchestra.run.smithi006:> import subprocess 2023-11-24T06:16:21.318 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-24T06:16:21.318 DEBUG:teuthology.orchestra.run.smithi006:> def _find_admin_socket(client_name): 2023-11-24T06:16:21.318 DEBUG:teuthology.orchestra.run.smithi006:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T06:16:21.318 DEBUG:teuthology.orchestra.run.smithi006:> files = glob.glob(asok_path) 2023-11-24T06:16:21.318 DEBUG:teuthology.orchestra.run.smithi006:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T06:16:21.318 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-24T06:16:21.319 DEBUG:teuthology.orchestra.run.smithi006:> # Given a non-glob path, it better be there 2023-11-24T06:16:21.319 DEBUG:teuthology.orchestra.run.smithi006:> if "*" not in asok_path: 2023-11-24T06:16:21.319 DEBUG:teuthology.orchestra.run.smithi006:> assert(len(files) == 1) 2023-11-24T06:16:21.319 DEBUG:teuthology.orchestra.run.smithi006:> return files[0] 2023-11-24T06:16:21.319 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-24T06:16:21.319 DEBUG:teuthology.orchestra.run.smithi006:> for f in files: 2023-11-24T06:16:21.319 DEBUG:teuthology.orchestra.run.smithi006:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:16:21.319 DEBUG:teuthology.orchestra.run.smithi006:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:16:21.319 DEBUG:teuthology.orchestra.run.smithi006:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:16:21.319 DEBUG:teuthology.orchestra.run.smithi006:> contents = proc_f.read() 2023-11-24T06:16:21.320 DEBUG:teuthology.orchestra.run.smithi006:> if mountpoint in contents: 2023-11-24T06:16:21.320 DEBUG:teuthology.orchestra.run.smithi006:> return f 2023-11-24T06:16:21.320 DEBUG:teuthology.orchestra.run.smithi006:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:16:21.320 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-24T06:16:21.320 DEBUG:teuthology.orchestra.run.smithi006:> print(_find_admin_socket("client.1")) 2023-11-24T06:16:21.320 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-24T06:16:21.522 INFO:teuthology.orchestra.run.smithi006.stdout:/var/run/ceph/ceph-client.1.93466.asok 2023-11-24T06:16:21.527 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:16:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:16:21.540 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.93466.asok 2023-11-24T06:16:21.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:21.541 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.93466.asok status 2023-11-24T06:16:21.792 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-24T06:16:21.792 INFO:teuthology.orchestra.run.smithi006.stdout: "metadata": { 2023-11-24T06:16:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:16:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:16:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: "entity_id": "1", 2023-11-24T06:16:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": "smithi006", 2023-11-24T06:16:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T06:16:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: "pid": "93466", 2023-11-24T06:16:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: "root": "/" 2023-11-24T06:16:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-24T06:16:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_count": 0, 2023-11-24T06:16:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_pinned_count": 0, 2023-11-24T06:16:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4442, 2023-11-24T06:16:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: "inst": { 2023-11-24T06:16:21.793 INFO:teuthology.orchestra.run.smithi006.stdout: "name": { 2023-11-24T06:16:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "client", 2023-11-24T06:16:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: "num": 4442 2023-11-24T06:16:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-24T06:16:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-24T06:16:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-24T06:16:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:16:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3339755277 2023-11-24T06:16:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-24T06:16:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-24T06:16:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-24T06:16:21.794 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-24T06:16:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:16:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3339755277 2023-11-24T06:16:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-24T06:16:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: "inst_str": "client.4442 192.168.0.1:0/3339755277", 2023-11-24T06:16:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: "addr_str": "192.168.0.1:0/3339755277", 2023-11-24T06:16:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: "inode_count": 1, 2023-11-24T06:16:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_epoch": 8, 2023-11-24T06:16:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch": 22, 2023-11-24T06:16:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:16:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: "blocklisted": false, 2023-11-24T06:16:21.795 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs" 2023-11-24T06:16:21.795 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-24T06:16:21.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:16:22.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.041+0000 7f408a66e640 1 Processor -- start 2023-11-24T06:16:22.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.041+0000 7f408a66e640 1 -- start start 2023-11-24T06:16:22.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.042+0000 7f408a66e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f408415d5b0 0x7f40840d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:22.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.042+0000 7f408a66e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f40840b1650 0x7f40840d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:22.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.042+0000 7f408a66e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40840cf0c0 con 0x7f40840d14a0 2023-11-24T06:16:22.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.043+0000 7f408a66e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40840b01b0 con 0x7f40840b1650 2023-11-24T06:16:22.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.043+0000 7f408a66e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408415d990 con 0x7f408415d5b0 2023-11-24T06:16:22.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.043+0000 7f408966c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f408415d5b0 0x7f40840d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:22.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.043+0000 7f4089e6d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f40840b1650 0x7f40840d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:22.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.043+0000 7f408966c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f408415d5b0 0x7f40840d3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:43444/0 (socket says 172.21.15.5:43444) 2023-11-24T06:16:22.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.043+0000 7f408966c640 1 -- 172.21.15.5:0/4028522116 learned_addr learned my addr 172.21.15.5:0/4028522116 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:22.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.044+0000 7f408966c640 1 -- 172.21.15.5:0/4028522116 >> v2:172.21.15.6:3300/0 conn(0x7f40840b1650 msgr2=0x7f40840d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:22.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.044+0000 7f408966c640 1 --2- 172.21.15.5:0/4028522116 >> v2:172.21.15.6:3300/0 conn(0x7f40840b1650 0x7f40840d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:22.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.044+0000 7f408966c640 1 -- 172.21.15.5:0/4028522116 >> v1:172.21.15.6:6789/0 conn(0x7f40840d14a0 legacy=0x7f40840cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:22.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.044+0000 7f408966c640 1 -- 172.21.15.5:0/4028522116 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40840d1e40 con 0x7f408415d5b0 2023-11-24T06:16:22.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.046+0000 7f408966c640 1 --2- 172.21.15.5:0/4028522116 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f408415d5b0 0x7f40840d3990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4080009e50 tx=0x7f408002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6225daa6bdd6148c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:22.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.046+0000 7f407bfff640 1 -- 172.21.15.5:0/4028522116 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408002cb00 con 0x7f408415d5b0 2023-11-24T06:16:22.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.046+0000 7f407bfff640 1 -- 172.21.15.5:0/4028522116 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408002cc60 con 0x7f408415d5b0 2023-11-24T06:16:22.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.046+0000 7f407bfff640 1 -- 172.21.15.5:0/4028522116 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40800343f0 con 0x7f408415d5b0 2023-11-24T06:16:22.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.047+0000 7f408a66e640 1 -- 172.21.15.5:0/4028522116 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f408415d5b0 msgr2=0x7f40840d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:22.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.047+0000 7f408a66e640 1 --2- 172.21.15.5:0/4028522116 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f408415d5b0 0x7f40840d3990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4080009e50 tx=0x7f408002bf60 comp rx=0 tx=0).stop 2023-11-24T06:16:22.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.047+0000 7f408a66e640 1 -- 172.21.15.5:0/4028522116 shutdown_connections 2023-11-24T06:16:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.047+0000 7f408a66e640 1 --2- 172.21.15.5:0/4028522116 >> v2:172.21.15.6:3300/0 conn(0x7f40840b1650 0x7f40840d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.047+0000 7f408a66e640 1 --2- 172.21.15.5:0/4028522116 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f408415d5b0 0x7f40840d3990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.047+0000 7f408a66e640 1 -- 172.21.15.5:0/4028522116 >> 172.21.15.5:0/4028522116 conn(0x7f408409c3b0 msgr2=0x7f40840d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.047+0000 7f408a66e640 1 -- 172.21.15.5:0/4028522116 shutdown_connections 2023-11-24T06:16:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.047+0000 7f408a66e640 1 -- 172.21.15.5:0/4028522116 wait complete. 2023-11-24T06:16:22.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.049+0000 7f408a66e640 1 Processor -- start 2023-11-24T06:16:22.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.049+0000 7f408a66e640 1 -- start start 2023-11-24T06:16:22.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.050+0000 7f408a66e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40840b1650 0x7f40840b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:22.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.050+0000 7f408a66e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40840d14a0 0x7f40840b7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:22.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.050+0000 7f408a66e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f408415d5b0 0x7f408415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:22.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.050+0000 7f408a66e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40840c9af0 con 0x7f40840b1650 2023-11-24T06:16:22.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.050+0000 7f408a66e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40840c9810 con 0x7f408415d5b0 2023-11-24T06:16:22.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.050+0000 7f408a66e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40840b82c0 con 0x7f40840d14a0 2023-11-24T06:16:22.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.050+0000 7f408966c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40840b1650 0x7f40840b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:22.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.051+0000 7f408966c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40840b1650 0x7f40840b8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50922/0 (socket says 172.21.15.5:50922) 2023-11-24T06:16:22.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.051+0000 7f408966c640 1 -- 172.21.15.5:0/827112879 learned_addr learned my addr 172.21.15.5:0/827112879 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:22.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.051+0000 7f4089e6d640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f408415d5b0 0x7f408415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:22.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.051+0000 7f4088e6b640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40840d14a0 0x7f40840b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:22.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.051+0000 7f408966c640 1 -- 172.21.15.5:0/827112879 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40840d14a0 msgr2=0x7f40840b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:22.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.051+0000 7f408966c640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40840d14a0 0x7f40840b7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:22.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.051+0000 7f408966c640 1 -- 172.21.15.5:0/827112879 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f408415d5b0 msgr2=0x7f408415aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:22.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.051+0000 7f408966c640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f408415d5b0 0x7f408415aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:22.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.051+0000 7f408966c640 1 -- 172.21.15.5:0/827112879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4080009b30 con 0x7f40840b1650 2023-11-24T06:16:22.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.051+0000 7f4089e6d640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f408415d5b0 0x7f408415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:22.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.051+0000 7f4088e6b640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40840d14a0 0x7f40840b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:22.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.051+0000 7f408966c640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40840b1650 0x7f40840b8ac0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4080000f80 tx=0x7f4080004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:22.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.052+0000 7f4079ffb640 1 -- 172.21.15.5:0/827112879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408003e450 con 0x7f40840b1650 2023-11-24T06:16:22.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.052+0000 7f4079ffb640 1 -- 172.21.15.5:0/827112879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408003e5d0 con 0x7f40840b1650 2023-11-24T06:16:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.052+0000 7f408a66e640 1 -- 172.21.15.5:0/827112879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40840b85a0 con 0x7f40840b1650 2023-11-24T06:16:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.052+0000 7f4079ffb640 1 -- 172.21.15.5:0/827112879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408003e980 con 0x7f40840b1650 2023-11-24T06:16:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.052+0000 7f408a66e640 1 -- 172.21.15.5:0/827112879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40840b8080 con 0x7f40840b1650 2023-11-24T06:16:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.053+0000 7f408a66e640 1 -- 172.21.15.5:0/827112879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408404e840 con 0x7f40840b1650 2023-11-24T06:16:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.055+0000 7f4079ffb640 1 -- 172.21.15.5:0/827112879 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f408003eae0 con 0x7f40840b1650 2023-11-24T06:16:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.055+0000 7f4079ffb640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f406806ae40 0x7f406806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.056+0000 7f4079ffb640 1 -- 172.21.15.5:0/827112879 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40800ac120 con 0x7f40840b1650 2023-11-24T06:16:22.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.056+0000 7f4088e6b640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f406806ae40 0x7f406806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:22.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.056+0000 7f4088e6b640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f406806ae40 0x7f406806d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f408406b590 tx=0x7f4074009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:22.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.060+0000 7f4079ffb640 1 -- 172.21.15.5:0/827112879 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40800771d0 con 0x7f40840b1650 2023-11-24T06:16:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.297+0000 7f408a66e640 1 -- 172.21.15.5:0/827112879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f408415e4e0 con 0x7f40840b1650 2023-11-24T06:16:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.297+0000 7f4079ffb640 1 -- 172.21.15.5:0/827112879 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f408007a7f0 con 0x7f40840b1650 2023-11-24T06:16:22.299 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:16:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.300+0000 7f408a66e640 1 -- 172.21.15.5:0/827112879 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f406806ae40 msgr2=0x7f406806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.300+0000 7f408a66e640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f406806ae40 0x7f406806d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f408406b590 tx=0x7f4074009040 comp rx=0 tx=0).stop 2023-11-24T06:16:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.300+0000 7f408a66e640 1 -- 172.21.15.5:0/827112879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40840b1650 msgr2=0x7f40840b8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.300+0000 7f408a66e640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40840b1650 0x7f40840b8ac0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4080000f80 tx=0x7f4080004a40 comp rx=0 tx=0).stop 2023-11-24T06:16:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.300+0000 7f408a66e640 1 -- 172.21.15.5:0/827112879 shutdown_connections 2023-11-24T06:16:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.300+0000 7f408a66e640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f406806ae40 0x7f406806d300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.300+0000 7f408a66e640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f40840d14a0 0x7f40840b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.300+0000 7f408a66e640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f408415d5b0 0x7f408415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.300+0000 7f408a66e640 1 --2- 172.21.15.5:0/827112879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40840b1650 0x7f40840b8ac0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.300+0000 7f408a66e640 1 -- 172.21.15.5:0/827112879 >> 172.21.15.5:0/827112879 conn(0x7f408409c3b0 msgr2=0x7f408415ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.300+0000 7f408a66e640 1 -- 172.21.15.5:0/827112879 shutdown_connections 2023-11-24T06:16:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:22.300+0000 7f408a66e640 1 -- 172.21.15.5:0/827112879 wait complete. 2023-11-24T06:16:22.321 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:16:22.321 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:16:22.321 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2023-11-24T06:16:22.321 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-24T06:16:22.321 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T06:16:22.322 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:16:22.322 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:16:22.322 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:16:22.322 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-24T06:16:22.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:22.322 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2023-11-24T06:16:22.387 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:16:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:16:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:16:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-24T06:16:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:22.388 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:16:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:22.388 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f0 2023-11-24T06:16:22.389 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:16:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f1 2023-11-24T06:16:22.389 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:16:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f0 2023-11-24T06:16:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T06:16:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 42317sec preferred_lft 42317sec 2023-11-24T06:16:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link noprefixroute 2023-11-24T06:16:22.389 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:22.389 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:16:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f1 2023-11-24T06:16:22.390 INFO:teuthology.orchestra.run.smithi006.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:16:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 9a:18:c2:c1:49:59 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:16:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:16:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::248e:b2ff:fe4b:5ed7/64 scope link 2023-11-24T06:16:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:22.390 INFO:teuthology.orchestra.run.smithi006.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:16:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 9a:18:c2:c1:49:59 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:16:22.390 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::9818:c2ff:fec1:4959/64 scope link 2023-11-24T06:16:22.391 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:22.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:22.392 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:16:22.490 INFO:teuthology.orchestra.run.smithi006.stdout:1 2023-11-24T06:16:22.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:22.495 DEBUG:teuthology.orchestra.run.smithi006:> route 2023-11-24T06:16:22.554 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2023-11-24T06:16:22.554 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:16:22.554 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T06:16:22.554 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T06:16:22.554 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:16:22.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:22.555 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-24T06:16:22.555 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-24T06:16:22.555 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T06:16:22.555 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T06:16:22.555 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T06:16:22.555 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-24T06:16:22.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:22.751 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-24T06:16:22.807 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:16:22.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:22.809 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list-id 2023-11-24T06:16:22.859 INFO:teuthology.orchestra.run.smithi006.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-24T06:16:22.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:22.860 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-24T06:16:22.861 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-24T06:16:22.861 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:16:22.861 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-24T06:16:22.861 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-24T06:16:23.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:23.003 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T06:16:23.103 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:16:23.103 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:16:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:16:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-24T06:16:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:23.104 INFO:teuthology.orchestra.run.smithi006.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:16:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 92:30:98:34:67:b8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:16:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:16:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::9030:98ff:fe34:67b8/64 scope link 2023-11-24T06:16:23.104 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:23.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:23.108 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T06:16:23.209 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:16:23.210 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:16:23.210 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:16:23.210 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:23.210 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-24T06:16:23.210 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:23.210 INFO:teuthology.orchestra.run.smithi006.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:16:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 92:30:98:34:67:b8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T06:16:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T06:16:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::9030:98ff:fe34:67b8/64 scope link 2023-11-24T06:16:23.211 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:16:23.214 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-24T06:16:23.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:23.214 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-24T06:16:23.214 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-24T06:16:23.214 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-24T06:16:23.215 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T06:16:23.215 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-24T06:16:23.215 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-24T06:16:23.215 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-24T06:16:23.215 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-24T06:16:23.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:23.486 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-24T06:16:23.487 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-24T06:16:23.487 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.1 up 2023-11-24T06:16:23.487 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T06:16:23.487 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-24T06:16:23.662 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-24T06:16:23.662 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:16:23.662 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-24T06:16:23.711 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-24T06:16:23.712 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:16:23.712 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-24T06:16:23.765 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-24T06:16:23.862 DEBUG:teuthology.orchestra.run.smithi006:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:16:23.921 INFO:teuthology.orchestra.run.smithi006.stdout:/proc 2023-11-24T06:16:23.921 INFO:teuthology.orchestra.run.smithi006.stdout:/sys 2023-11-24T06:16:23.922 INFO:teuthology.orchestra.run.smithi006.stdout:/dev 2023-11-24T06:16:23.922 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/security 2023-11-24T06:16:23.922 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/shm 2023-11-24T06:16:23.922 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/pts 2023-11-24T06:16:23.922 INFO:teuthology.orchestra.run.smithi006.stdout:/run 2023-11-24T06:16:23.922 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/cgroup 2023-11-24T06:16:23.922 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/pstore 2023-11-24T06:16:23.922 INFO:teuthology.orchestra.run.smithi006.stdout:/ 2023-11-24T06:16:23.922 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/selinux 2023-11-24T06:16:23.922 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:16:23.923 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/hugepages 2023-11-24T06:16:23.923 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/mqueue 2023-11-24T06:16:23.923 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/debug 2023-11-24T06:16:23.923 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/tracing 2023-11-24T06:16:23.923 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/fuse/connections 2023-11-24T06:16:23.923 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/config 2023-11-24T06:16:23.923 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:16:23.923 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:16:23.923 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:16:23.924 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:16:23.924 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/1000 2023-11-24T06:16:23.924 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph 2023-11-24T06:16:23.924 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:16:23.924 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:16:23.924 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:16:23.924 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:16:23.924 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns 2023-11-24T06:16:23.924 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:16:23.924 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:16:23.924 INFO:teuthology.orchestra.run.smithi006.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T06:16:23.924 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:16:23.924 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:16:23.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:23.926 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-24T06:16:23.980 INFO:teuthology.orchestra.run.smithi006.stdout:41 2023-11-24T06:16:23.980 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-11-24T06:16:23.980 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-11-24T06:16:23.986 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-24T06:16:24.063 DEBUG:teuthology.orchestra.run.smithi006:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:16:24.107 INFO:teuthology.orchestra.run.smithi006.stdout:/proc 2023-11-24T06:16:24.107 INFO:teuthology.orchestra.run.smithi006.stdout:/sys 2023-11-24T06:16:24.107 INFO:teuthology.orchestra.run.smithi006.stdout:/dev 2023-11-24T06:16:24.107 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/security 2023-11-24T06:16:24.107 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/shm 2023-11-24T06:16:24.107 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/pts 2023-11-24T06:16:24.107 INFO:teuthology.orchestra.run.smithi006.stdout:/run 2023-11-24T06:16:24.107 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/cgroup 2023-11-24T06:16:24.108 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/pstore 2023-11-24T06:16:24.108 INFO:teuthology.orchestra.run.smithi006.stdout:/ 2023-11-24T06:16:24.108 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/selinux 2023-11-24T06:16:24.108 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:16:24.108 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/hugepages 2023-11-24T06:16:24.108 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/mqueue 2023-11-24T06:16:24.108 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/debug 2023-11-24T06:16:24.108 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/tracing 2023-11-24T06:16:24.108 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/fuse/connections 2023-11-24T06:16:24.108 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/config 2023-11-24T06:16:24.108 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:16:24.108 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:16:24.109 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:16:24.109 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:16:24.109 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/1000 2023-11-24T06:16:24.109 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph 2023-11-24T06:16:24.109 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:16:24.109 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:16:24.109 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:16:24.109 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:16:24.109 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns 2023-11-24T06:16:24.109 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:16:24.109 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:16:24.110 INFO:teuthology.orchestra.run.smithi006.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T06:16:24.110 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:16:24.110 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:16:24.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:24.111 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-24T06:16:24.176 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi006.stderr:2023-11-24T06:16:24.174+0000 7fae9f9f9480 -1 init, newargv = 0x55f2efcf0f80 newargc=15 2023-11-24T06:16:24.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi006.stderr:ceph-fuse[93918]: starting ceph client 2023-11-24T06:16:24.186 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi006.stderr:ceph-fuse[93918]: starting fuse 2023-11-24T06:16:24.200 INFO:teuthology.orchestra.run.smithi006.stdout:41 2023-11-24T06:16:24.200 INFO:teuthology.orchestra.run.smithi006.stdout:42 2023-11-24T06:16:24.200 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 42] 2023-11-24T06:16:24.200 DEBUG:teuthology.orchestra.run.smithi006:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:16:24.200 DEBUG:teuthology.orchestra.run.smithi006:> import glob 2023-11-24T06:16:24.200 DEBUG:teuthology.orchestra.run.smithi006:> import re 2023-11-24T06:16:24.201 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-24T06:16:24.201 DEBUG:teuthology.orchestra.run.smithi006:> import subprocess 2023-11-24T06:16:24.201 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-24T06:16:24.201 DEBUG:teuthology.orchestra.run.smithi006:> def _find_admin_socket(client_name): 2023-11-24T06:16:24.201 DEBUG:teuthology.orchestra.run.smithi006:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-24T06:16:24.201 DEBUG:teuthology.orchestra.run.smithi006:> files = glob.glob(asok_path) 2023-11-24T06:16:24.201 DEBUG:teuthology.orchestra.run.smithi006:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-24T06:16:24.201 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-24T06:16:24.201 DEBUG:teuthology.orchestra.run.smithi006:> # Given a non-glob path, it better be there 2023-11-24T06:16:24.201 DEBUG:teuthology.orchestra.run.smithi006:> if "*" not in asok_path: 2023-11-24T06:16:24.201 DEBUG:teuthology.orchestra.run.smithi006:> assert(len(files) == 1) 2023-11-24T06:16:24.202 DEBUG:teuthology.orchestra.run.smithi006:> return files[0] 2023-11-24T06:16:24.202 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-24T06:16:24.202 DEBUG:teuthology.orchestra.run.smithi006:> for f in files: 2023-11-24T06:16:24.202 DEBUG:teuthology.orchestra.run.smithi006:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:16:24.202 DEBUG:teuthology.orchestra.run.smithi006:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:16:24.202 DEBUG:teuthology.orchestra.run.smithi006:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:16:24.202 DEBUG:teuthology.orchestra.run.smithi006:> contents = proc_f.read() 2023-11-24T06:16:24.202 DEBUG:teuthology.orchestra.run.smithi006:> if mountpoint in contents: 2023-11-24T06:16:24.203 DEBUG:teuthology.orchestra.run.smithi006:> return f 2023-11-24T06:16:24.203 DEBUG:teuthology.orchestra.run.smithi006:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:16:24.203 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-24T06:16:24.203 DEBUG:teuthology.orchestra.run.smithi006:> print(_find_admin_socket("client.3")) 2023-11-24T06:16:24.203 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-24T06:16:24.422 INFO:teuthology.orchestra.run.smithi006.stdout:/var/run/ceph/ceph-client.3.93918.asok 2023-11-24T06:16:24.427 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:16:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:16:24.441 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.93918.asok 2023-11-24T06:16:24.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:24.441 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.93918.asok status 2023-11-24T06:16:24.690 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-24T06:16:24.690 INFO:teuthology.orchestra.run.smithi006.stdout: "metadata": { 2023-11-24T06:16:24.691 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:16:24.691 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:16:24.691 INFO:teuthology.orchestra.run.smithi006.stdout: "entity_id": "3", 2023-11-24T06:16:24.691 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": "smithi006", 2023-11-24T06:16:24.691 INFO:teuthology.orchestra.run.smithi006.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-24T06:16:24.691 INFO:teuthology.orchestra.run.smithi006.stdout: "pid": "93918", 2023-11-24T06:16:24.691 INFO:teuthology.orchestra.run.smithi006.stdout: "root": "/" 2023-11-24T06:16:24.691 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-24T06:16:24.691 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_count": 0, 2023-11-24T06:16:24.691 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_pinned_count": 0, 2023-11-24T06:16:24.691 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 4597, 2023-11-24T06:16:24.691 INFO:teuthology.orchestra.run.smithi006.stdout: "inst": { 2023-11-24T06:16:24.691 INFO:teuthology.orchestra.run.smithi006.stdout: "name": { 2023-11-24T06:16:24.692 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "client", 2023-11-24T06:16:24.692 INFO:teuthology.orchestra.run.smithi006.stdout: "num": 4597 2023-11-24T06:16:24.692 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-24T06:16:24.692 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-24T06:16:24.692 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-24T06:16:24.692 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:16:24.692 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3170526685 2023-11-24T06:16:24.692 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-24T06:16:24.692 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-24T06:16:24.692 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-24T06:16:24.692 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-24T06:16:24.692 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.2:0", 2023-11-24T06:16:24.692 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 3170526685 2023-11-24T06:16:24.693 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-24T06:16:24.693 INFO:teuthology.orchestra.run.smithi006.stdout: "inst_str": "client.4597 192.168.0.2:0/3170526685", 2023-11-24T06:16:24.693 INFO:teuthology.orchestra.run.smithi006.stdout: "addr_str": "192.168.0.2:0/3170526685", 2023-11-24T06:16:24.693 INFO:teuthology.orchestra.run.smithi006.stdout: "inode_count": 1, 2023-11-24T06:16:24.693 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_epoch": 8, 2023-11-24T06:16:24.693 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch": 22, 2023-11-24T06:16:24.693 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:16:24.693 INFO:teuthology.orchestra.run.smithi006.stdout: "blocklisted": false, 2023-11-24T06:16:24.693 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs" 2023-11-24T06:16:24.693 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-24T06:16:24.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:24.705 DEBUG:teuthology.orchestra.run.smithi005:> stat --file-system '--printf=%T 2023-11-24T06:16:24.705 DEBUG:teuthology.orchestra.run.smithi005:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T06:16:24.770 INFO:teuthology.orchestra.run.smithi005.stdout:fuseblk 2023-11-24T06:16:24.770 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T06:16:24.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:24.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:16:24.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:24.881 DEBUG:teuthology.orchestra.run.smithi005:> stat --file-system '--printf=%T 2023-11-24T06:16:24.881 DEBUG:teuthology.orchestra.run.smithi005:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-24T06:16:24.942 INFO:teuthology.orchestra.run.smithi005.stdout:fuseblk 2023-11-24T06:16:24.942 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-24T06:16:24.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:24.942 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-24T06:16:25.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:25.053 DEBUG:teuthology.orchestra.run.smithi006:> stat --file-system '--printf=%T 2023-11-24T06:16:25.053 DEBUG:teuthology.orchestra.run.smithi006:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T06:16:25.119 INFO:teuthology.orchestra.run.smithi006.stdout:fuseblk 2023-11-24T06:16:25.119 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T06:16:25.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:25.120 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:16:25.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:25.223 DEBUG:teuthology.orchestra.run.smithi006:> stat --file-system '--printf=%T 2023-11-24T06:16:25.223 DEBUG:teuthology.orchestra.run.smithi006:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-24T06:16:25.284 INFO:teuthology.orchestra.run.smithi006.stdout:fuseblk 2023-11-24T06:16:25.284 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-24T06:16:25.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:25.284 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-24T06:16:25.387 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-24T06:16:25.406 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:25.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.642+0000 7fc8adeb4640 1 Processor -- start 2023-11-24T06:16:25.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.642+0000 7fc8adeb4640 1 -- start start 2023-11-24T06:16:25.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.643+0000 7fc8adeb4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8a80cf050 0x7fc8a80cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:25.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.644+0000 7fc8adeb4640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a8140900 con 0x7fc8a813dfd0 2023-11-24T06:16:25.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.644+0000 7fc8adeb4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a815fa70 con 0x7fc8a80cd7b0 2023-11-24T06:16:25.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.644+0000 7fc8adeb4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a815c580 con 0x7fc8a80cf050 2023-11-24T06:16:25.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.644+0000 7fc8aceb2640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc8a813dfd0 0x7fc8a815be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35464/0 (socket says 172.21.15.5:35464) 2023-11-24T06:16:25.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.644+0000 7fc8aceb2640 1 -- 172.21.15.5:0/2301529685 learned_addr learned my addr 172.21.15.5:0/2301529685 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:25.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.644+0000 7fc8a7fff640 1 --2- 172.21.15.5:0/2301529685 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8a80cf050 0x7fc8a80cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:25.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.644+0000 7fc8a77fe640 1 -- 172.21.15.5:0/2301529685 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225509993 0 0) 0x7fc8a8140900 con 0x7fc8a813dfd0 2023-11-24T06:16:25.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a77fe640 1 -- 172.21.15.5:0/2301529685 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc898003520 con 0x7fc8a813dfd0 2023-11-24T06:16:25.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a77fe640 1 -- 172.21.15.5:0/2301529685 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1935116332 0 0) 0x7fc898003520 con 0x7fc8a813dfd0 2023-11-24T06:16:25.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a77fe640 1 -- 172.21.15.5:0/2301529685 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8a80cf050 msgr2=0x7fc8a80cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:25.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a77fe640 1 --2- 172.21.15.5:0/2301529685 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8a80cf050 0x7fc8a80cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:25.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a77fe640 1 -- 172.21.15.5:0/2301529685 >> v1:172.21.15.6:6789/0 conn(0x7fc8a80cd7b0 legacy=0x7fc8a80cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:25.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a77fe640 1 -- 172.21.15.5:0/2301529685 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a815ee60 con 0x7fc8a813dfd0 2023-11-24T06:16:25.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a77fe640 1 -- 172.21.15.5:0/2301529685 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fc89c001670 con 0x7fc8a813dfd0 2023-11-24T06:16:25.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a77fe640 1 -- 172.21.15.5:0/2301529685 >> v1:172.21.15.5:6789/0 conn(0x7fc8a813dfd0 legacy=0x7fc8a815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:25.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a77fe640 1 --2- 172.21.15.5:0/2301529685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc898003a10 0x7fc898003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:25.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a77fe640 1 --2- 172.21.15.5:0/2301529685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8980047c0 0x7fc898004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:25.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a77fe640 1 --2- 172.21.15.5:0/2301529685 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8a80cf050 0x7fc898005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:25.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a77fe640 1 -- 172.21.15.5:0/2301529685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a815c580 con 0x7fc8980047c0 2023-11-24T06:16:25.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a77fe640 1 -- 172.21.15.5:0/2301529685 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89800ba30 con 0x7fc898003a10 2023-11-24T06:16:25.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a77fe640 1 -- 172.21.15.5:0/2301529685 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89800bbc0 con 0x7fc8a80cf050 2023-11-24T06:16:25.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.645+0000 7fc8a7fff640 1 --2- 172.21.15.5:0/2301529685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8980047c0 0x7fc898004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:25.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.646+0000 7fc8aceb2640 1 --2- 172.21.15.5:0/2301529685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc898003a10 0x7fc898003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.646+0000 7fc8a7fff640 1 -- 172.21.15.5:0/2301529685 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8a80cf050 msgr2=0x7fc898005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.646+0000 7fc8a7fff640 1 --2- 172.21.15.5:0/2301529685 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8a80cf050 0x7fc898005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.646+0000 7fc8a7fff640 1 -- 172.21.15.5:0/2301529685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc898003a10 msgr2=0x7fc898003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.646+0000 7fc8a7fff640 1 --2- 172.21.15.5:0/2301529685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc898003a10 0x7fc898003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.646+0000 7fc8a7fff640 1 -- 172.21.15.5:0/2301529685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc898003520 con 0x7fc8980047c0 2023-11-24T06:16:25.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.646+0000 7fc8aceb2640 1 --2- 172.21.15.5:0/2301529685 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc898003a10 0x7fc898003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.647+0000 7fc8a7fff640 1 --2- 172.21.15.5:0/2301529685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8980047c0 0x7fc898004c40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc894009ac0 tx=0x7fc8980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbec17b362786f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.647+0000 7fc8a77fe640 1 -- 172.21.15.5:0/2301529685 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89402c680 con 0x7fc8980047c0 2023-11-24T06:16:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.647+0000 7fc8a77fe640 1 -- 172.21.15.5:0/2301529685 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89402c7e0 con 0x7fc8980047c0 2023-11-24T06:16:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.647+0000 7fc8a77fe640 1 -- 172.21.15.5:0/2301529685 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89402cb70 con 0x7fc8980047c0 2023-11-24T06:16:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.648+0000 7fc8adeb4640 1 -- 172.21.15.5:0/2301529685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8980047c0 msgr2=0x7fc898004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.648+0000 7fc8adeb4640 1 --2- 172.21.15.5:0/2301529685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8980047c0 0x7fc898004c40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc894009ac0 tx=0x7fc8980057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.648+0000 7fc8ad6b3640 1 -- 172.21.15.5:0/2301529685 reap_dead start 2023-11-24T06:16:25.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.648+0000 7fc8adeb4640 1 -- 172.21.15.5:0/2301529685 shutdown_connections 2023-11-24T06:16:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.648+0000 7fc8adeb4640 1 -- 172.21.15.5:0/2301529685 >> 172.21.15.5:0/2301529685 conn(0x7fc8a806a860 msgr2=0x7fc8a806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.648+0000 7fc8adeb4640 1 -- 172.21.15.5:0/2301529685 shutdown_connections 2023-11-24T06:16:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.648+0000 7fc8adeb4640 1 -- 172.21.15.5:0/2301529685 wait complete. 2023-11-24T06:16:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.650+0000 7fc8adeb4640 1 Processor -- start 2023-11-24T06:16:25.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.650+0000 7fc8adeb4640 1 -- start start 2023-11-24T06:16:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.651+0000 7fc8adeb4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8980047c0 0x7fc8a815dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.651+0000 7fc8adeb4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a8138c30 0x7fc8a81390b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.651+0000 7fc8adeb4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8a815a630 0x7fc8a815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.651+0000 7fc8adeb4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a81395f0 con 0x7fc8a8138c30 2023-11-24T06:16:25.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.651+0000 7fc8adeb4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a8153ef0 con 0x7fc8980047c0 2023-11-24T06:16:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.651+0000 7fc8adeb4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a81541d0 con 0x7fc8a815a630 2023-11-24T06:16:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.652+0000 7fc8a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a8138c30 0x7fc8a81390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.652+0000 7fc8a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a8138c30 0x7fc8a81390b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40338/0 (socket says 172.21.15.5:40338) 2023-11-24T06:16:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.652+0000 7fc8a7fff640 1 -- 172.21.15.5:0/3526654344 learned_addr learned my addr 172.21.15.5:0/3526654344 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.652+0000 7fc8aceb2640 1 --2- 172.21.15.5:0/3526654344 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8980047c0 0x7fc8a815dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.652+0000 7fc8a7fff640 1 -- 172.21.15.5:0/3526654344 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8a815a630 msgr2=0x7fc8a815aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.652+0000 7fc8a7fff640 1 --2- 172.21.15.5:0/3526654344 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8a815a630 0x7fc8a815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.652+0000 7fc8a7fff640 1 -- 172.21.15.5:0/3526654344 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8980047c0 msgr2=0x7fc8a815dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.652+0000 7fc8a7fff640 1 --2- 172.21.15.5:0/3526654344 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8980047c0 0x7fc8a815dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.652+0000 7fc8a7fff640 1 -- 172.21.15.5:0/3526654344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a81544b0 con 0x7fc8a8138c30 2023-11-24T06:16:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.652+0000 7fc8aceb2640 1 --2- 172.21.15.5:0/3526654344 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8980047c0 0x7fc8a815dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:25.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.653+0000 7fc8a7fff640 1 --2- 172.21.15.5:0/3526654344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a8138c30 0x7fc8a81390b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc89402c220 tx=0x7fc894004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.653+0000 7fc8a57fa640 1 -- 172.21.15.5:0/3526654344 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894003810 con 0x7fc8a8138c30 2023-11-24T06:16:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.653+0000 7fc8a57fa640 1 -- 172.21.15.5:0/3526654344 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc894003970 con 0x7fc8a8138c30 2023-11-24T06:16:25.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.653+0000 7fc8adeb4640 1 -- 172.21.15.5:0/3526654344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a8154790 con 0x7fc8a8138c30 2023-11-24T06:16:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.653+0000 7fc8a57fa640 1 -- 172.21.15.5:0/3526654344 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894003ce0 con 0x7fc8a8138c30 2023-11-24T06:16:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.653+0000 7fc8adeb4640 1 -- 172.21.15.5:0/3526654344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a815b000 con 0x7fc8a8138c30 2023-11-24T06:16:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.654+0000 7fc8adeb4640 1 -- 172.21.15.5:0/3526654344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a804e840 con 0x7fc8a8138c30 2023-11-24T06:16:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.655+0000 7fc8a57fa640 1 -- 172.21.15.5:0/3526654344 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc894040450 con 0x7fc8a8138c30 2023-11-24T06:16:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.656+0000 7fc8a57fa640 1 --2- 172.21.15.5:0/3526654344 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc88c06ae40 0x7fc88c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.656+0000 7fc8a57fa640 1 -- 172.21.15.5:0/3526654344 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8940ab510 con 0x7fc8a8138c30 2023-11-24T06:16:25.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.656+0000 7fc8aceb2640 1 --2- 172.21.15.5:0/3526654344 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc88c06ae40 0x7fc88c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:25.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.657+0000 7fc8aceb2640 1 --2- 172.21.15.5:0/3526654344 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc88c06ae40 0x7fc88c06d300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc89c000f60 tx=0x7fc89c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.661+0000 7fc8a57fa640 1 -- 172.21.15.5:0/3526654344 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc894076520 con 0x7fc8a8138c30 2023-11-24T06:16:25.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.855+0000 7fc8adeb4640 1 -- 172.21.15.5:0/3526654344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8a80b3b60 con 0x7fc8a8138c30 2023-11-24T06:16:25.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.857+0000 7fc8a57fa640 1 -- 172.21.15.5:0/3526654344 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16361 (secure 0 0 0) 0x7fc8940044e0 con 0x7fc8a8138c30 2023-11-24T06:16:25.859 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:25.859 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:52.779059+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:25.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.860+0000 7fc8adeb4640 1 -- 172.21.15.5:0/3526654344 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc88c06ae40 msgr2=0x7fc88c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:25.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.860+0000 7fc8adeb4640 1 --2- 172.21.15.5:0/3526654344 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc88c06ae40 0x7fc88c06d300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc89c000f60 tx=0x7fc89c009040 comp rx=0 tx=0).stop 2023-11-24T06:16:25.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.860+0000 7fc8adeb4640 1 -- 172.21.15.5:0/3526654344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a8138c30 msgr2=0x7fc8a81390b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:25.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.860+0000 7fc8adeb4640 1 --2- 172.21.15.5:0/3526654344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a8138c30 0x7fc8a81390b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc89402c220 tx=0x7fc894004730 comp rx=0 tx=0).stop 2023-11-24T06:16:25.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.860+0000 7fc8adeb4640 1 -- 172.21.15.5:0/3526654344 shutdown_connections 2023-11-24T06:16:25.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.860+0000 7fc8adeb4640 1 --2- 172.21.15.5:0/3526654344 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc88c06ae40 0x7fc88c06d300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:25.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.860+0000 7fc8adeb4640 1 --2- 172.21.15.5:0/3526654344 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8a815a630 0x7fc8a815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:25.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.860+0000 7fc8adeb4640 1 --2- 172.21.15.5:0/3526654344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a8138c30 0x7fc8a81390b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:25.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.860+0000 7fc8adeb4640 1 --2- 172.21.15.5:0/3526654344 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8980047c0 0x7fc8a815dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:25.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.860+0000 7fc8adeb4640 1 -- 172.21.15.5:0/3526654344 >> 172.21.15.5:0/3526654344 conn(0x7fc8a806a860 msgr2=0x7fc8a8140260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:25.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.861+0000 7fc8adeb4640 1 -- 172.21.15.5:0/3526654344 shutdown_connections 2023-11-24T06:16:25.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:25.861+0000 7fc8adeb4640 1 -- 172.21.15.5:0/3526654344 wait complete. 2023-11-24T06:16:25.882 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:25.882 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:26.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.098+0000 7fa308a2d640 1 Processor -- start 2023-11-24T06:16:26.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.098+0000 7fa308a2d640 1 -- start start 2023-11-24T06:16:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.099+0000 7fa308a2d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa304160bd0 0x7fa3040b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.099+0000 7fa308a2d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3040b3210 0x7fa304167290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.099+0000 7fa308a2d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3040b23e0 con 0x7fa3040b48d0 2023-11-24T06:16:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.100+0000 7fa308a2d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3040cdc70 con 0x7fa304160bd0 2023-11-24T06:16:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.100+0000 7fa308a2d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040b2870 con 0x7fa3040b3210 2023-11-24T06:16:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.100+0000 7fa30259c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa304160bd0 0x7fa3040b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.100+0000 7fa301d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa3040b48d0 0x7fa3040b2b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35470/0 (socket says 172.21.15.5:35470) 2023-11-24T06:16:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.100+0000 7fa301d9b640 1 -- 172.21.15.5:0/3892116782 learned_addr learned my addr 172.21.15.5:0/3892116782 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.100+0000 7fa302d9d640 1 --2- 172.21.15.5:0/3892116782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3040b3210 0x7fa304167290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.100+0000 7fa30259c640 1 -- 172.21.15.5:0/3892116782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3040b3210 msgr2=0x7fa304167290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.100+0000 7fa30259c640 1 --2- 172.21.15.5:0/3892116782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3040b3210 0x7fa304167290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.100+0000 7fa30159a640 1 -- 172.21.15.5:0/3892116782 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1569506633 0 0) 0x7fa3040b23e0 con 0x7fa3040b48d0 2023-11-24T06:16:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.100+0000 7fa30259c640 1 -- 172.21.15.5:0/3892116782 >> v1:172.21.15.5:6789/0 conn(0x7fa3040b48d0 legacy=0x7fa3040b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.100+0000 7fa30259c640 1 -- 172.21.15.5:0/3892116782 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3040d41b0 con 0x7fa304160bd0 2023-11-24T06:16:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.102+0000 7fa30259c640 1 --2- 172.21.15.5:0/3892116782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa304160bd0 0x7fa3040b1ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009d90 tx=0x7fa2f402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fdbd6591461850 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.102+0000 7fa30159a640 1 -- 172.21.15.5:0/3892116782 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f402cbf0 con 0x7fa304160bd0 2023-11-24T06:16:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.102+0000 7fa30159a640 1 -- 172.21.15.5:0/3892116782 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f402cd50 con 0x7fa304160bd0 2023-11-24T06:16:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.102+0000 7fa30159a640 1 -- 172.21.15.5:0/3892116782 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4034660 con 0x7fa304160bd0 2023-11-24T06:16:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.102+0000 7fa308a2d640 1 -- 172.21.15.5:0/3892116782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa304160bd0 msgr2=0x7fa3040b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.102+0000 7fa308a2d640 1 --2- 172.21.15.5:0/3892116782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa304160bd0 0x7fa3040b1ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4009d90 tx=0x7fa2f402be40 comp rx=0 tx=0).stop 2023-11-24T06:16:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.103+0000 7fa308a2d640 1 -- 172.21.15.5:0/3892116782 shutdown_connections 2023-11-24T06:16:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.103+0000 7fa308a2d640 1 --2- 172.21.15.5:0/3892116782 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3040b3210 0x7fa304167290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.103+0000 7fa308a2d640 1 --2- 172.21.15.5:0/3892116782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa304160bd0 0x7fa3040b1ea0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.103+0000 7fa308a2d640 1 -- 172.21.15.5:0/3892116782 >> 172.21.15.5:0/3892116782 conn(0x7fa30409c5a0 msgr2=0x7fa3040d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.103+0000 7fa308a2d640 1 -- 172.21.15.5:0/3892116782 shutdown_connections 2023-11-24T06:16:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.103+0000 7fa308a2d640 1 -- 172.21.15.5:0/3892116782 wait complete. 2023-11-24T06:16:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.105+0000 7fa308a2d640 1 Processor -- start 2023-11-24T06:16:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.105+0000 7fa308a2d640 1 -- start start 2023-11-24T06:16:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.106+0000 7fa308a2d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3040b3210 0x7fa30414f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.106+0000 7fa308a2d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3040b48d0 0x7fa3040b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.106+0000 7fa308a2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa304160bd0 0x7fa3040b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.106+0000 7fa308a2d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30414fbe0 con 0x7fa304160bd0 2023-11-24T06:16:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.106+0000 7fa308a2d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040ca8d0 con 0x7fa3040b3210 2023-11-24T06:16:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.106+0000 7fa308a2d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040cabb0 con 0x7fa3040b48d0 2023-11-24T06:16:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.106+0000 7fa302d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa304160bd0 0x7fa3040b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.106+0000 7fa302d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa304160bd0 0x7fa3040b7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40364/0 (socket says 172.21.15.5:40364) 2023-11-24T06:16:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.106+0000 7fa302d9d640 1 -- 172.21.15.5:0/2518871091 learned_addr learned my addr 172.21.15.5:0/2518871091 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.106+0000 7fa302d9d640 1 -- 172.21.15.5:0/2518871091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3040b48d0 msgr2=0x7fa3040b7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.106+0000 7fa301d9b640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3040b48d0 0x7fa3040b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.106+0000 7fa30259c640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3040b3210 0x7fa30414f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.107+0000 7fa302d9d640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3040b48d0 0x7fa3040b7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.107+0000 7fa302d9d640 1 -- 172.21.15.5:0/2518871091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3040b3210 msgr2=0x7fa30414f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.107+0000 7fa302d9d640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3040b3210 0x7fa30414f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.107+0000 7fa302d9d640 1 -- 172.21.15.5:0/2518871091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f4009a70 con 0x7fa304160bd0 2023-11-24T06:16:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.107+0000 7fa301d9b640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3040b48d0 0x7fa3040b7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.107+0000 7fa30259c640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3040b3210 0x7fa30414f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:26.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.107+0000 7fa302d9d640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa304160bd0 0x7fa3040b7f60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00d800 tx=0x7fa2fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:26.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.107+0000 7fa2faffd640 1 -- 172.21.15.5:0/2518871091 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc00b6c0 con 0x7fa304160bd0 2023-11-24T06:16:26.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.107+0000 7fa2faffd640 1 -- 172.21.15.5:0/2518871091 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc00b840 con 0x7fa304160bd0 2023-11-24T06:16:26.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.107+0000 7fa2faffd640 1 -- 172.21.15.5:0/2518871091 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc00bbf0 con 0x7fa304160bd0 2023-11-24T06:16:26.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.107+0000 7fa308a2d640 1 -- 172.21.15.5:0/2518871091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3040cae90 con 0x7fa304160bd0 2023-11-24T06:16:26.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.107+0000 7fa308a2d640 1 -- 172.21.15.5:0/2518871091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3040cb450 con 0x7fa304160bd0 2023-11-24T06:16:26.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.109+0000 7fa308a2d640 1 -- 172.21.15.5:0/2518871091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3040d2610 con 0x7fa304160bd0 2023-11-24T06:16:26.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.110+0000 7fa2faffd640 1 -- 172.21.15.5:0/2518871091 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa2fc00bd50 con 0x7fa304160bd0 2023-11-24T06:16:26.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.110+0000 7fa2faffd640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa2e406ae40 0x7fa2e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:26.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.111+0000 7fa2faffd640 1 -- 172.21.15.5:0/2518871091 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2fc089c40 con 0x7fa304160bd0 2023-11-24T06:16:26.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.111+0000 7fa30259c640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa2e406ae40 0x7fa2e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:26.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.112+0000 7fa30259c640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa2e406ae40 0x7fa2e406d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4036000 tx=0x7fa2f4030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:26.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.115+0000 7fa2faffd640 1 -- 172.21.15.5:0/2518871091 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa2fc054c50 con 0x7fa304160bd0 2023-11-24T06:16:26.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.308+0000 7fa308a2d640 1 -- 172.21.15.5:0/2518871091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3040cb670 con 0x7fa304160bd0 2023-11-24T06:16:26.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.308+0000 7fa2faffd640 1 -- 172.21.15.5:0/2518871091 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2fc058270 con 0x7fa304160bd0 2023-11-24T06:16:26.310 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:26.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.311+0000 7fa308a2d640 1 -- 172.21.15.5:0/2518871091 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa2e406ae40 msgr2=0x7fa2e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:26.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.311+0000 7fa308a2d640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa2e406ae40 0x7fa2e406d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4036000 tx=0x7fa2f4030000 comp rx=0 tx=0).stop 2023-11-24T06:16:26.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.311+0000 7fa308a2d640 1 -- 172.21.15.5:0/2518871091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa304160bd0 msgr2=0x7fa3040b7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:26.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.311+0000 7fa308a2d640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa304160bd0 0x7fa3040b7f60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00d800 tx=0x7fa2fc00dcc0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.311+0000 7fa308a2d640 1 -- 172.21.15.5:0/2518871091 shutdown_connections 2023-11-24T06:16:26.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.311+0000 7fa308a2d640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa2e406ae40 0x7fa2e406d300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.311+0000 7fa308a2d640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3040b48d0 0x7fa3040b7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.311+0000 7fa308a2d640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa304160bd0 0x7fa3040b7f60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.311+0000 7fa308a2d640 1 --2- 172.21.15.5:0/2518871091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3040b3210 0x7fa30414f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.311+0000 7fa308a2d640 1 -- 172.21.15.5:0/2518871091 >> 172.21.15.5:0/2518871091 conn(0x7fa30409c5a0 msgr2=0x7fa3040cf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.311+0000 7fa308a2d640 1 -- 172.21.15.5:0/2518871091 shutdown_connections 2023-11-24T06:16:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.311+0000 7fa308a2d640 1 -- 172.21.15.5:0/2518871091 wait complete. 2023-11-24T06:16:26.333 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.566+0000 7fdb22999640 1 Processor -- start 2023-11-24T06:16:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.566+0000 7fdb22999640 1 -- start start 2023-11-24T06:16:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.568+0000 7fdb22999640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdb1c144770 0x7fdb1c15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.568+0000 7fdb22999640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdb1c15f0a0 0x7fdb1c0ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.568+0000 7fdb22999640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb1c15f930 con 0x7fdb1c0b35a0 2023-11-24T06:16:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.568+0000 7fdb22999640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb1c0b43b0 con 0x7fdb1c144770 2023-11-24T06:16:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.568+0000 7fdb22999640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb1c15f510 con 0x7fdb1c15f0a0 2023-11-24T06:16:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.568+0000 7fdb21997640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdb1c144770 0x7fdb1c15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.568+0000 7fdb21997640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdb1c144770 0x7fdb1c15eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40368/0 (socket says 172.21.15.5:40368) 2023-11-24T06:16:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.568+0000 7fdb21997640 1 -- 172.21.15.5:0/1416585395 learned_addr learned my addr 172.21.15.5:0/1416585395 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.569+0000 7fdb22198640 1 --2- 172.21.15.5:0/1416585395 >> v2:172.21.15.6:3300/0 conn(0x7fdb1c15f0a0 0x7fdb1c0ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.569+0000 7fdb21997640 1 -- 172.21.15.5:0/1416585395 >> v2:172.21.15.6:3300/0 conn(0x7fdb1c15f0a0 msgr2=0x7fdb1c0ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.569+0000 7fdb21997640 1 --2- 172.21.15.5:0/1416585395 >> v2:172.21.15.6:3300/0 conn(0x7fdb1c15f0a0 0x7fdb1c0ce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.569+0000 7fdb21997640 1 -- 172.21.15.5:0/1416585395 >> v1:172.21.15.6:6789/0 conn(0x7fdb1c0b35a0 legacy=0x7fdb1c06c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.569+0000 7fdb21997640 1 -- 172.21.15.5:0/1416585395 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1c0ce9f0 con 0x7fdb1c144770 2023-11-24T06:16:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.570+0000 7fdb21997640 1 --2- 172.21.15.5:0/1416585395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1c144770 0x7fdb1c15eb60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0700e0 tx=0x7fdb1802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdcea75abd631341 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.571+0000 7fdb20995640 1 -- 172.21.15.5:0/1416585395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb1802ca80 con 0x7fdb1c144770 2023-11-24T06:16:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.571+0000 7fdb20995640 1 -- 172.21.15.5:0/1416585395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb1802cbe0 con 0x7fdb1c144770 2023-11-24T06:16:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.571+0000 7fdb20995640 1 -- 172.21.15.5:0/1416585395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb180343f0 con 0x7fdb1c144770 2023-11-24T06:16:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.571+0000 7fdb22999640 1 -- 172.21.15.5:0/1416585395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1c144770 msgr2=0x7fdb1c15eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.571+0000 7fdb22999640 1 --2- 172.21.15.5:0/1416585395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1c144770 0x7fdb1c15eb60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c0700e0 tx=0x7fdb1802bee0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.572+0000 7fdb22999640 1 -- 172.21.15.5:0/1416585395 shutdown_connections 2023-11-24T06:16:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.572+0000 7fdb22999640 1 --2- 172.21.15.5:0/1416585395 >> v2:172.21.15.6:3300/0 conn(0x7fdb1c15f0a0 0x7fdb1c0ce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.572+0000 7fdb22999640 1 --2- 172.21.15.5:0/1416585395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1c144770 0x7fdb1c15eb60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.572+0000 7fdb22999640 1 -- 172.21.15.5:0/1416585395 >> 172.21.15.5:0/1416585395 conn(0x7fdb1c0cec10 msgr2=0x7fdb1c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.572+0000 7fdb22999640 1 -- 172.21.15.5:0/1416585395 shutdown_connections 2023-11-24T06:16:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.572+0000 7fdb22999640 1 -- 172.21.15.5:0/1416585395 wait complete. 2023-11-24T06:16:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.574+0000 7fdb22999640 1 Processor -- start 2023-11-24T06:16:26.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.574+0000 7fdb22999640 1 -- start start 2023-11-24T06:16:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.575+0000 7fdb22999640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb1c0b35a0 0x7fdb1c1f1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.575+0000 7fdb22999640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1c144770 0x7fdb1c13b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.575+0000 7fdb22999640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb1c15f0a0 0x7fdb1c13be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.575+0000 7fdb22999640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c13ef40 con 0x7fdb1c144770 2023-11-24T06:16:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.575+0000 7fdb22999640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c13f150 con 0x7fdb1c0b35a0 2023-11-24T06:16:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.575+0000 7fdb22999640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c13f430 con 0x7fdb1c15f0a0 2023-11-24T06:16:26.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.576+0000 7fdb21196640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1c144770 0x7fdb1c13b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:26.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.576+0000 7fdb21196640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1c144770 0x7fdb1c13b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40372/0 (socket says 172.21.15.5:40372) 2023-11-24T06:16:26.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.576+0000 7fdb21196640 1 -- 172.21.15.5:0/549698032 learned_addr learned my addr 172.21.15.5:0/549698032 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:26.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.576+0000 7fdb22198640 1 --2- 172.21.15.5:0/549698032 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb1c15f0a0 0x7fdb1c13be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:26.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.576+0000 7fdb21997640 1 --2- 172.21.15.5:0/549698032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb1c0b35a0 0x7fdb1c1f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:26.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.576+0000 7fdb21196640 1 -- 172.21.15.5:0/549698032 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb1c15f0a0 msgr2=0x7fdb1c13be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:26.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.576+0000 7fdb21196640 1 --2- 172.21.15.5:0/549698032 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb1c15f0a0 0x7fdb1c13be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.576+0000 7fdb21196640 1 -- 172.21.15.5:0/549698032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb1c0b35a0 msgr2=0x7fdb1c1f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:26.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.576+0000 7fdb21196640 1 --2- 172.21.15.5:0/549698032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb1c0b35a0 0x7fdb1c1f1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.576+0000 7fdb21196640 1 -- 172.21.15.5:0/549698032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb18009b30 con 0x7fdb1c144770 2023-11-24T06:16:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.576+0000 7fdb21196640 1 --2- 172.21.15.5:0/549698032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1c144770 0x7fdb1c13b920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00b3f0 tx=0x7fdb0c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.577+0000 7fdb127fc640 1 -- 172.21.15.5:0/549698032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c011830 con 0x7fdb1c144770 2023-11-24T06:16:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.577+0000 7fdb127fc640 1 -- 172.21.15.5:0/549698032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0c011990 con 0x7fdb1c144770 2023-11-24T06:16:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.577+0000 7fdb22999640 1 -- 172.21.15.5:0/549698032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1c13f710 con 0x7fdb1c144770 2023-11-24T06:16:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.577+0000 7fdb127fc640 1 -- 172.21.15.5:0/549698032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c011d40 con 0x7fdb1c144770 2023-11-24T06:16:26.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.577+0000 7fdb22999640 1 -- 172.21.15.5:0/549698032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1c13ff10 con 0x7fdb1c144770 2023-11-24T06:16:26.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.578+0000 7fdb22999640 1 -- 172.21.15.5:0/549698032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1c04e840 con 0x7fdb1c144770 2023-11-24T06:16:26.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.579+0000 7fdb127fc640 1 -- 172.21.15.5:0/549698032 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb0c01a400 con 0x7fdb1c144770 2023-11-24T06:16:26.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.580+0000 7fdb127fc640 1 --2- 172.21.15.5:0/549698032 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdb0006ae40 0x7fdb0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:26.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.580+0000 7fdb127fc640 1 -- 172.21.15.5:0/549698032 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb0c089460 con 0x7fdb1c144770 2023-11-24T06:16:26.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.580+0000 7fdb21997640 1 --2- 172.21.15.5:0/549698032 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdb0006ae40 0x7fdb0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:26.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.581+0000 7fdb21997640 1 --2- 172.21.15.5:0/549698032 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdb0006ae40 0x7fdb0006d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb18035000 tx=0x7fdb18030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:26.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.585+0000 7fdb127fc640 1 -- 172.21.15.5:0/549698032 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb0c054500 con 0x7fdb1c144770 2023-11-24T06:16:26.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.781+0000 7fdb22999640 1 -- 172.21.15.5:0/549698032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb1c0b2460 con 0x7fdb1c144770 2023-11-24T06:16:26.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.782+0000 7fdb127fc640 1 -- 172.21.15.5:0/549698032 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb0c057b20 con 0x7fdb1c144770 2023-11-24T06:16:26.784 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.785+0000 7fdb22999640 1 -- 172.21.15.5:0/549698032 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdb0006ae40 msgr2=0x7fdb0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.785+0000 7fdb22999640 1 --2- 172.21.15.5:0/549698032 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdb0006ae40 0x7fdb0006d300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb18035000 tx=0x7fdb18030000 comp rx=0 tx=0).stop 2023-11-24T06:16:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.785+0000 7fdb22999640 1 -- 172.21.15.5:0/549698032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1c144770 msgr2=0x7fdb1c13b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.785+0000 7fdb22999640 1 --2- 172.21.15.5:0/549698032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1c144770 0x7fdb1c13b920 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00b3f0 tx=0x7fdb0c00b8b0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.786+0000 7fdb22999640 1 -- 172.21.15.5:0/549698032 shutdown_connections 2023-11-24T06:16:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.786+0000 7fdb22999640 1 --2- 172.21.15.5:0/549698032 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdb0006ae40 0x7fdb0006d300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.786+0000 7fdb22999640 1 --2- 172.21.15.5:0/549698032 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb1c15f0a0 0x7fdb1c13be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.786+0000 7fdb22999640 1 --2- 172.21.15.5:0/549698032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1c144770 0x7fdb1c13b920 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.786+0000 7fdb22999640 1 --2- 172.21.15.5:0/549698032 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb1c0b35a0 0x7fdb1c1f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.786+0000 7fdb22999640 1 -- 172.21.15.5:0/549698032 >> 172.21.15.5:0/549698032 conn(0x7fdb1c0cec10 msgr2=0x7fdb1c15da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.786+0000 7fdb22999640 1 -- 172.21.15.5:0/549698032 shutdown_connections 2023-11-24T06:16:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:26.786+0000 7fdb22999640 1 -- 172.21.15.5:0/549698032 wait complete. 2023-11-24T06:16:26.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.034+0000 7f4d4445c640 1 Processor -- start 2023-11-24T06:16:27.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.034+0000 7f4d4445c640 1 -- start start 2023-11-24T06:16:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.036+0000 7f4d4445c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4d3c06cad0 0x7f4d3c06e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.036+0000 7f4d4445c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4d3c072280 0x7f4d3c0b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.036+0000 7f4d4445c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3c071780 con 0x7f4d3c160b10 2023-11-24T06:16:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.036+0000 7f4d4445c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3c0b2320 con 0x7f4d3c06cad0 2023-11-24T06:16:27.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.036+0000 7f4d4445c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3c06df30 con 0x7f4d3c072280 2023-11-24T06:16:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.036+0000 7f4d419f7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4d3c06cad0 0x7f4d3c06e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.037+0000 7f4d419f7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4d3c06cad0 0x7f4d3c06e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40374/0 (socket says 172.21.15.5:40374) 2023-11-24T06:16:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.037+0000 7f4d419f7640 1 -- 172.21.15.5:0/2381599365 learned_addr learned my addr 172.21.15.5:0/2381599365 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.037+0000 7f4d429f9640 1 --2- 172.21.15.5:0/2381599365 >> v2:172.21.15.6:3300/0 conn(0x7f4d3c072280 0x7f4d3c0b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.037+0000 7f4d419f7640 1 -- 172.21.15.5:0/2381599365 >> v2:172.21.15.6:3300/0 conn(0x7f4d3c072280 msgr2=0x7f4d3c0b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.037+0000 7f4d419f7640 1 --2- 172.21.15.5:0/2381599365 >> v2:172.21.15.6:3300/0 conn(0x7f4d3c072280 0x7f4d3c0b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.037+0000 7f4d419f7640 1 -- 172.21.15.5:0/2381599365 >> v1:172.21.15.6:6789/0 conn(0x7f4d3c160b10 legacy=0x7f4d3c06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:27.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.037+0000 7f4d419f7640 1 -- 172.21.15.5:0/2381599365 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c0711c0 con 0x7f4d3c06cad0 2023-11-24T06:16:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.038+0000 7f4d419f7640 1 --2- 172.21.15.5:0/2381599365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d3c06cad0 0x7f4d3c06e420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0b3fe0 tx=0x7f4d2c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b4c5ce2f7b9e47 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.039+0000 7f4d411f6640 1 -- 172.21.15.5:0/2381599365 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0317e0 con 0x7f4d3c06cad0 2023-11-24T06:16:27.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.039+0000 7f4d411f6640 1 -- 172.21.15.5:0/2381599365 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c031940 con 0x7f4d3c06cad0 2023-11-24T06:16:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.039+0000 7f4d411f6640 1 -- 172.21.15.5:0/2381599365 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c031cd0 con 0x7f4d3c06cad0 2023-11-24T06:16:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.039+0000 7f4d4445c640 1 -- 172.21.15.5:0/2381599365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d3c06cad0 msgr2=0x7f4d3c06e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.039+0000 7f4d4445c640 1 --2- 172.21.15.5:0/2381599365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d3c06cad0 0x7f4d3c06e420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c0b3fe0 tx=0x7f4d2c02b880 comp rx=0 tx=0).stop 2023-11-24T06:16:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.039+0000 7f4d4445c640 1 -- 172.21.15.5:0/2381599365 shutdown_connections 2023-11-24T06:16:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.039+0000 7f4d4445c640 1 --2- 172.21.15.5:0/2381599365 >> v2:172.21.15.6:3300/0 conn(0x7f4d3c072280 0x7f4d3c0b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.039+0000 7f4d4445c640 1 --2- 172.21.15.5:0/2381599365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d3c06cad0 0x7f4d3c06e420 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.039+0000 7f4d4445c640 1 -- 172.21.15.5:0/2381599365 >> 172.21.15.5:0/2381599365 conn(0x7f4d3c0b0ba0 msgr2=0x7f4d3c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.039+0000 7f4d4445c640 1 -- 172.21.15.5:0/2381599365 shutdown_connections 2023-11-24T06:16:27.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.039+0000 7f4d4445c640 1 -- 172.21.15.5:0/2381599365 wait complete. 2023-11-24T06:16:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.041+0000 7f4d4445c640 1 Processor -- start 2023-11-24T06:16:27.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.041+0000 7f4d4445c640 1 -- start start 2023-11-24T06:16:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.042+0000 7f4d4445c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d3c06cad0 0x7f4d3c1f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.042+0000 7f4d4445c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d3c072280 0x7f4d3c0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.042+0000 7f4d4445c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d3c160b10 0x7f4d3c099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.042+0000 7f4d4445c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c151410 con 0x7f4d3c160b10 2023-11-24T06:16:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.042+0000 7f4d4445c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0ce940 con 0x7f4d3c072280 2023-11-24T06:16:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.042+0000 7f4d4445c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c1f2c20 con 0x7f4d3c06cad0 2023-11-24T06:16:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.043+0000 7f4d429f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d3c160b10 0x7f4d3c099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.043+0000 7f4d429f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d3c160b10 0x7f4d3c099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40376/0 (socket says 172.21.15.5:40376) 2023-11-24T06:16:27.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.043+0000 7f4d429f9640 1 -- 172.21.15.5:0/565852437 learned_addr learned my addr 172.21.15.5:0/565852437 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.043+0000 7f4d419f7640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d3c072280 0x7f4d3c0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.043+0000 7f4d421f8640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d3c06cad0 0x7f4d3c1f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.043+0000 7f4d429f9640 1 -- 172.21.15.5:0/565852437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d3c06cad0 msgr2=0x7f4d3c1f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.043+0000 7f4d429f9640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d3c06cad0 0x7f4d3c1f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.043+0000 7f4d429f9640 1 -- 172.21.15.5:0/565852437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d3c072280 msgr2=0x7f4d3c0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.043+0000 7f4d429f9640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d3c072280 0x7f4d3c0cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.043+0000 7f4d429f9640 1 -- 172.21.15.5:0/565852437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c0094d0 con 0x7f4d3c160b10 2023-11-24T06:16:27.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.043+0000 7f4d421f8640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d3c06cad0 0x7f4d3c1f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.043+0000 7f4d429f9640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d3c160b10 0x7f4d3c099a00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4d34007e40 tx=0x7f4d3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.044+0000 7f4d32ffd640 1 -- 172.21.15.5:0/565852437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34014620 con 0x7f4d3c160b10 2023-11-24T06:16:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.044+0000 7f4d32ffd640 1 -- 172.21.15.5:0/565852437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d340147a0 con 0x7f4d3c160b10 2023-11-24T06:16:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.044+0000 7f4d32ffd640 1 -- 172.21.15.5:0/565852437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34014b50 con 0x7f4d3c160b10 2023-11-24T06:16:27.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.044+0000 7f4d4445c640 1 -- 172.21.15.5:0/565852437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c158b90 con 0x7f4d3c160b10 2023-11-24T06:16:27.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.044+0000 7f4d4445c640 1 -- 172.21.15.5:0/565852437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c0ce660 con 0x7f4d3c160b10 2023-11-24T06:16:27.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.045+0000 7f4d4445c640 1 -- 172.21.15.5:0/565852437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3c09d200 con 0x7f4d3c160b10 2023-11-24T06:16:27.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.046+0000 7f4d32ffd640 1 -- 172.21.15.5:0/565852437 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d34014cb0 con 0x7f4d3c160b10 2023-11-24T06:16:27.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.047+0000 7f4d32ffd640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4d2006ae40 0x7f4d2006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.047+0000 7f4d32ffd640 1 -- 172.21.15.5:0/565852437 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d34089f80 con 0x7f4d3c160b10 2023-11-24T06:16:27.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.047+0000 7f4d421f8640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4d2006ae40 0x7f4d2006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.048+0000 7f4d421f8640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4d2006ae40 0x7f4d2006d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800a830 tx=0x7f4d38008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:27.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.052+0000 7f4d32ffd640 1 -- 172.21.15.5:0/565852437 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d34055020 con 0x7f4d3c160b10 2023-11-24T06:16:27.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.249+0000 7f4d4445c640 1 -- 172.21.15.5:0/565852437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d3c04e920 con 0x7f4d3c160b10 2023-11-24T06:16:27.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.250+0000 7f4d32ffd640 1 -- 172.21.15.5:0/565852437 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d34058640 con 0x7f4d3c160b10 2023-11-24T06:16:27.252 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:27.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.252+0000 7f4d4445c640 1 -- 172.21.15.5:0/565852437 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4d2006ae40 msgr2=0x7f4d2006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:27.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.252+0000 7f4d4445c640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4d2006ae40 0x7f4d2006d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800a830 tx=0x7f4d38008040 comp rx=0 tx=0).stop 2023-11-24T06:16:27.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.252+0000 7f4d4445c640 1 -- 172.21.15.5:0/565852437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d3c160b10 msgr2=0x7f4d3c099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:27.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.252+0000 7f4d4445c640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d3c160b10 0x7f4d3c099a00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4d34007e40 tx=0x7f4d3400e4b0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.252+0000 7f4d4445c640 1 -- 172.21.15.5:0/565852437 shutdown_connections 2023-11-24T06:16:27.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.252+0000 7f4d4445c640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4d2006ae40 0x7f4d2006d300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.252+0000 7f4d4445c640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d3c160b10 0x7f4d3c099a00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.252+0000 7f4d4445c640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4d3c072280 0x7f4d3c0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.252+0000 7f4d4445c640 1 --2- 172.21.15.5:0/565852437 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4d3c06cad0 0x7f4d3c1f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.252+0000 7f4d4445c640 1 -- 172.21.15.5:0/565852437 >> 172.21.15.5:0/565852437 conn(0x7f4d3c0b0ba0 msgr2=0x7f4d3c06c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:27.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.253+0000 7f4d4445c640 1 -- 172.21.15.5:0/565852437 shutdown_connections 2023-11-24T06:16:27.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.253+0000 7f4d4445c640 1 -- 172.21.15.5:0/565852437 wait complete. 2023-11-24T06:16:27.274 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:27.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.492+0000 7f0be8a2e640 1 Processor -- start 2023-11-24T06:16:27.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.492+0000 7f0be8a2e640 1 -- start start 2023-11-24T06:16:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.494+0000 7f0be8a2e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0be4143570 0x7f0be40cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.494+0000 7f0be8a2e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0be4143950 0x7f0be40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.494+0000 7f0be8a2e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be4071700 con 0x7f0be415e6b0 2023-11-24T06:16:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.494+0000 7f0be8a2e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be415b410 con 0x7f0be4143950 2023-11-24T06:16:27.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.494+0000 7f0be8a2e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be415b130 con 0x7f0be4143570 2023-11-24T06:16:27.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.494+0000 7f0be3fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0be415e6b0 0x7f0be415a4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35482/0 (socket says 172.21.15.5:35482) 2023-11-24T06:16:27.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.494+0000 7f0be3fff640 1 -- 172.21.15.5:0/3955408382 learned_addr learned my addr 172.21.15.5:0/3955408382 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:27.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.494+0000 7f0bdbfff640 1 --2- 172.21.15.5:0/3955408382 >> v2:172.21.15.5:3300/0 conn(0x7f0be4143950 0x7f0be40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.495+0000 7f0be37fe640 1 --2- 172.21.15.5:0/3955408382 >> v2:172.21.15.6:3300/0 conn(0x7f0be4143570 0x7f0be40cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.495+0000 7f0be2ffd640 1 -- 172.21.15.5:0/3955408382 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 97132049 0 0) 0x7f0bdc001490 con 0x7f0be415e6b0 2023-11-24T06:16:27.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.495+0000 7f0be2ffd640 1 -- 172.21.15.5:0/3955408382 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bd0003520 con 0x7f0be415e6b0 2023-11-24T06:16:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.495+0000 7f0bdbfff640 1 -- 172.21.15.5:0/3955408382 >> v2:172.21.15.6:3300/0 conn(0x7f0be4143570 msgr2=0x7f0be40cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.495+0000 7f0bdbfff640 1 --2- 172.21.15.5:0/3955408382 >> v2:172.21.15.6:3300/0 conn(0x7f0be4143570 0x7f0be40cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.495+0000 7f0bdbfff640 1 -- 172.21.15.5:0/3955408382 >> v1:172.21.15.5:6789/0 conn(0x7f0be415e6b0 legacy=0x7f0be415a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.495+0000 7f0bdbfff640 1 -- 172.21.15.5:0/3955408382 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be415dbd0 con 0x7f0be4143950 2023-11-24T06:16:27.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.496+0000 7f0bdbfff640 1 --2- 172.21.15.5:0/3955408382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0be4143950 0x7f0be40cf590 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0be4070c70 tx=0x7f0bcc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb717b71e103110 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.497+0000 7f0be2ffd640 1 -- 172.21.15.5:0/3955408382 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc02e070 con 0x7f0be4143950 2023-11-24T06:16:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.497+0000 7f0be2ffd640 1 -- 172.21.15.5:0/3955408382 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bcc0315a0 con 0x7f0be4143950 2023-11-24T06:16:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.497+0000 7f0be2ffd640 1 -- 172.21.15.5:0/3955408382 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc031930 con 0x7f0be4143950 2023-11-24T06:16:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.497+0000 7f0be8a2e640 1 -- 172.21.15.5:0/3955408382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0be4143950 msgr2=0x7f0be40cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:27.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.497+0000 7f0be8a2e640 1 --2- 172.21.15.5:0/3955408382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0be4143950 0x7f0be40cf590 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0be4070c70 tx=0x7f0bcc02b840 comp rx=0 tx=0).stop 2023-11-24T06:16:27.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.497+0000 7f0be8a2e640 1 -- 172.21.15.5:0/3955408382 shutdown_connections 2023-11-24T06:16:27.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.497+0000 7f0be8a2e640 1 --2- 172.21.15.5:0/3955408382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0be4143950 0x7f0be40cf590 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.497+0000 7f0be8a2e640 1 --2- 172.21.15.5:0/3955408382 >> v2:172.21.15.6:3300/0 conn(0x7f0be4143570 0x7f0be40cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.497+0000 7f0be8a2e640 1 -- 172.21.15.5:0/3955408382 >> 172.21.15.5:0/3955408382 conn(0x7f0be406a860 msgr2=0x7f0be406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:27.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.497+0000 7f0be8a2e640 1 -- 172.21.15.5:0/3955408382 shutdown_connections 2023-11-24T06:16:27.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.497+0000 7f0be8a2e640 1 -- 172.21.15.5:0/3955408382 wait complete. 2023-11-24T06:16:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.499+0000 7f0be8a2e640 1 Processor -- start 2023-11-24T06:16:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.499+0000 7f0be8a2e640 1 -- start start 2023-11-24T06:16:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.500+0000 7f0be8a2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0be4143570 0x7f0be413a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.500+0000 7f0be8a2e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0be4143950 0x7f0be41f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.500+0000 7f0be8a2e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0be415e6b0 0x7f0be406d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.500+0000 7f0be8a2e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be413b540 con 0x7f0be4143570 2023-11-24T06:16:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.500+0000 7f0be8a2e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be4141cf0 con 0x7f0be4143950 2023-11-24T06:16:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.500+0000 7f0be8a2e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be413a6b0 con 0x7f0be415e6b0 2023-11-24T06:16:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.500+0000 7f0be37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0be4143570 0x7f0be413a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.501+0000 7f0be37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0be4143570 0x7f0be413a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40396/0 (socket says 172.21.15.5:40396) 2023-11-24T06:16:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.501+0000 7f0be37fe640 1 -- 172.21.15.5:0/1346964235 learned_addr learned my addr 172.21.15.5:0/1346964235 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.501+0000 7f0bdbfff640 1 --2- 172.21.15.5:0/1346964235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0be4143950 0x7f0be41f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.501+0000 7f0be37fe640 1 -- 172.21.15.5:0/1346964235 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0be415e6b0 msgr2=0x7f0be406d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.501+0000 7f0be37fe640 1 --2- 172.21.15.5:0/1346964235 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0be415e6b0 0x7f0be406d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.501+0000 7f0be37fe640 1 -- 172.21.15.5:0/1346964235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0be4143950 msgr2=0x7f0be41f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.501+0000 7f0be37fe640 1 --2- 172.21.15.5:0/1346964235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0be4143950 0x7f0be41f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.501+0000 7f0be37fe640 1 -- 172.21.15.5:0/1346964235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc009400 con 0x7f0be4143570 2023-11-24T06:16:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.501+0000 7f0bdbfff640 1 --2- 172.21.15.5:0/1346964235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0be4143950 0x7f0be41f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:27.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.501+0000 7f0be37fe640 1 --2- 172.21.15.5:0/1346964235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0be4143570 0x7f0be413a170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400d5f0 tx=0x7f0bd400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.501+0000 7f0be0ff9640 1 -- 172.21.15.5:0/1346964235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd40125f0 con 0x7f0be4143570 2023-11-24T06:16:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.502+0000 7f0be0ff9640 1 -- 172.21.15.5:0/1346964235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd4012750 con 0x7f0be4143570 2023-11-24T06:16:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.502+0000 7f0be8a2e640 1 -- 172.21.15.5:0/1346964235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be41f66e0 con 0x7f0be4143570 2023-11-24T06:16:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.502+0000 7f0be0ff9640 1 -- 172.21.15.5:0/1346964235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd4012b00 con 0x7f0be4143570 2023-11-24T06:16:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.502+0000 7f0be8a2e640 1 -- 172.21.15.5:0/1346964235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be413aba0 con 0x7f0be4143570 2023-11-24T06:16:27.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.503+0000 7f0be8a2e640 1 -- 172.21.15.5:0/1346964235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be404e840 con 0x7f0be4143570 2023-11-24T06:16:27.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.504+0000 7f0be0ff9640 1 -- 172.21.15.5:0/1346964235 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0bd4012c60 con 0x7f0be4143570 2023-11-24T06:16:27.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.505+0000 7f0be0ff9640 1 --2- 172.21.15.5:0/1346964235 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0bc406ae40 0x7f0bc406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.505+0000 7f0be0ff9640 1 -- 172.21.15.5:0/1346964235 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bd4089a90 con 0x7f0be4143570 2023-11-24T06:16:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.505+0000 7f0bdbfff640 1 --2- 172.21.15.5:0/1346964235 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0bc406ae40 0x7f0bc406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.506+0000 7f0bdbfff640 1 --2- 172.21.15.5:0/1346964235 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0bc406ae40 0x7f0bc406d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc036000 tx=0x7f0bcc03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.509+0000 7f0be0ff9640 1 -- 172.21.15.5:0/1346964235 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0bd4054bd0 con 0x7f0be4143570 2023-11-24T06:16:27.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.696+0000 7f0be8a2e640 1 -- 172.21.15.5:0/1346964235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0be40b3910 con 0x7f0be4143570 2023-11-24T06:16:27.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.699+0000 7f0be0ff9640 1 -- 172.21.15.5:0/1346964235 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16361 (secure 0 0 0) 0x7f0bd40581f0 con 0x7f0be4143570 2023-11-24T06:16:27.701 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:27.701 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:52.779059+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:27.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.701+0000 7f0be8a2e640 1 -- 172.21.15.5:0/1346964235 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0bc406ae40 msgr2=0x7f0bc406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:27.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.701+0000 7f0be8a2e640 1 --2- 172.21.15.5:0/1346964235 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0bc406ae40 0x7f0bc406d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc036000 tx=0x7f0bcc03d040 comp rx=0 tx=0).stop 2023-11-24T06:16:27.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.701+0000 7f0be8a2e640 1 -- 172.21.15.5:0/1346964235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0be4143570 msgr2=0x7f0be413a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:27.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.701+0000 7f0be8a2e640 1 --2- 172.21.15.5:0/1346964235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0be4143570 0x7f0be413a170 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0bd400d5f0 tx=0x7f0bd400dab0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.701+0000 7f0be8a2e640 1 -- 172.21.15.5:0/1346964235 shutdown_connections 2023-11-24T06:16:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.701+0000 7f0be8a2e640 1 --2- 172.21.15.5:0/1346964235 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0bc406ae40 0x7f0bc406d300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.701+0000 7f0be8a2e640 1 --2- 172.21.15.5:0/1346964235 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0be415e6b0 0x7f0be406d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.701+0000 7f0be8a2e640 1 --2- 172.21.15.5:0/1346964235 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0be4143950 0x7f0be41f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.701+0000 7f0be8a2e640 1 --2- 172.21.15.5:0/1346964235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0be4143570 0x7f0be413a170 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.701+0000 7f0be8a2e640 1 -- 172.21.15.5:0/1346964235 >> 172.21.15.5:0/1346964235 conn(0x7f0be406a860 msgr2=0x7f0be40b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.701+0000 7f0be8a2e640 1 -- 172.21.15.5:0/1346964235 shutdown_connections 2023-11-24T06:16:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.701+0000 7f0be8a2e640 1 -- 172.21.15.5:0/1346964235 wait complete. 2023-11-24T06:16:27.721 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:27.722 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.945+0000 7f59d9cbb640 1 Processor -- start 2023-11-24T06:16:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.946+0000 7f59d9cbb640 1 -- start start 2023-11-24T06:16:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.947+0000 7f59d9cbb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59d4160bd0 0x7f59d40b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.947+0000 7f59d9cbb640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f59d40b48d0 0x7f59d40b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.947+0000 7f59d9cbb640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f59d40b3040 0x7f59d4166280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.947+0000 7f59d9cbb640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d40cdc70 con 0x7f59d40b3040 2023-11-24T06:16:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.947+0000 7f59d9cbb640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d40b2870 con 0x7f59d40b48d0 2023-11-24T06:16:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.947+0000 7f59d9cbb640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40b23e0 con 0x7f59d4160bd0 2023-11-24T06:16:27.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.948+0000 7f59d3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f59d40b3040 0x7f59d4166280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.948+0000 7f59d3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f59d40b3040 0x7f59d4166280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40410/0 (socket says 172.21.15.5:40410) 2023-11-24T06:16:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.948+0000 7f59d3fff640 1 -- 172.21.15.5:0/2834276714 learned_addr learned my addr 172.21.15.5:0/2834276714 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.948+0000 7f59d37fe640 1 --2- 172.21.15.5:0/2834276714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59d4160bd0 0x7f59d40b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.948+0000 7f59d2ffd640 1 --2- 172.21.15.5:0/2834276714 >> v2:172.21.15.6:3300/0 conn(0x7f59d40b48d0 0x7f59d40b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.948+0000 7f59d3fff640 1 -- 172.21.15.5:0/2834276714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59d4160bd0 msgr2=0x7f59d40b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.948+0000 7f59d3fff640 1 --2- 172.21.15.5:0/2834276714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59d4160bd0 0x7f59d40b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.948+0000 7f59d3fff640 1 -- 172.21.15.5:0/2834276714 >> v2:172.21.15.6:3300/0 conn(0x7f59d40b48d0 msgr2=0x7f59d40b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.948+0000 7f59d3fff640 1 --2- 172.21.15.5:0/2834276714 >> v2:172.21.15.6:3300/0 conn(0x7f59d40b48d0 0x7f59d40b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.948+0000 7f59d3fff640 1 -- 172.21.15.5:0/2834276714 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d40d41b0 con 0x7f59d40b3040 2023-11-24T06:16:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.950+0000 7f59d3fff640 1 --2- 172.21.15.5:0/2834276714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d40b3040 0x7f59d4166280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f59d408b970 tx=0x7f59cc02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2407fd1e8381d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.950+0000 7f59d27fc640 1 -- 172.21.15.5:0/2834276714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0317d0 con 0x7f59d40b3040 2023-11-24T06:16:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.950+0000 7f59d27fc640 1 -- 172.21.15.5:0/2834276714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc031930 con 0x7f59d40b3040 2023-11-24T06:16:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.950+0000 7f59d27fc640 1 -- 172.21.15.5:0/2834276714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc031cc0 con 0x7f59d40b3040 2023-11-24T06:16:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.951+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2834276714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d40b3040 msgr2=0x7f59d4166280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:27.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.951+0000 7f59d9cbb640 1 --2- 172.21.15.5:0/2834276714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d40b3040 0x7f59d4166280 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f59d408b970 tx=0x7f59cc02b870 comp rx=0 tx=0).stop 2023-11-24T06:16:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.951+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2834276714 shutdown_connections 2023-11-24T06:16:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.951+0000 7f59d9cbb640 1 --2- 172.21.15.5:0/2834276714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d40b3040 0x7f59d4166280 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.951+0000 7f59d9cbb640 1 --2- 172.21.15.5:0/2834276714 >> v2:172.21.15.6:3300/0 conn(0x7f59d40b48d0 0x7f59d40b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.951+0000 7f59d9cbb640 1 --2- 172.21.15.5:0/2834276714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59d4160bd0 0x7f59d40b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.951+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2834276714 >> 172.21.15.5:0/2834276714 conn(0x7f59d409c5a0 msgr2=0x7f59d40d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.951+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2834276714 shutdown_connections 2023-11-24T06:16:27.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.951+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2834276714 wait complete. 2023-11-24T06:16:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.952+0000 7f59d9cbb640 1 Processor -- start 2023-11-24T06:16:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.953+0000 7f59d9cbb640 1 -- start start 2023-11-24T06:16:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.953+0000 7f59d9cbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d40b3040 0x7f59d41543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.954+0000 7f59d9cbb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59d40b48d0 0x7f59d41526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.954+0000 7f59d9cbb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59d4160bd0 0x7f59d40c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.954+0000 7f59d9cbb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d40b5e60 con 0x7f59d40b3040 2023-11-24T06:16:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.954+0000 7f59d9cbb640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4152c10 con 0x7f59d4160bd0 2023-11-24T06:16:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.954+0000 7f59d9cbb640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d4154010 con 0x7f59d40b48d0 2023-11-24T06:16:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.954+0000 7f59d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d40b3040 0x7f59d41543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.954+0000 7f59d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d40b3040 0x7f59d41543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40424/0 (socket says 172.21.15.5:40424) 2023-11-24T06:16:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.954+0000 7f59d37fe640 1 -- 172.21.15.5:0/2965930749 learned_addr learned my addr 172.21.15.5:0/2965930749 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.954+0000 7f59d37fe640 1 -- 172.21.15.5:0/2965930749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59d40b48d0 msgr2=0x7f59d41526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.954+0000 7f59d37fe640 1 --2- 172.21.15.5:0/2965930749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59d40b48d0 0x7f59d41526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.954+0000 7f59d37fe640 1 -- 172.21.15.5:0/2965930749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59d4160bd0 msgr2=0x7f59d40c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.954+0000 7f59d37fe640 1 --2- 172.21.15.5:0/2965930749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59d4160bd0 0x7f59d40c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.954+0000 7f59d37fe640 1 -- 172.21.15.5:0/2965930749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc0094d0 con 0x7f59d40b3040 2023-11-24T06:16:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.954+0000 7f59d37fe640 1 --2- 172.21.15.5:0/2965930749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d40b3040 0x7f59d41543d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f59c4007e70 tx=0x7f59c400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.955+0000 7f59cbfff640 1 -- 172.21.15.5:0/2965930749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4014620 con 0x7f59d40b3040 2023-11-24T06:16:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.955+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2965930749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d40b60b0 con 0x7f59d40b3040 2023-11-24T06:16:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.955+0000 7f59cbfff640 1 -- 172.21.15.5:0/2965930749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c4014780 con 0x7f59d40b3040 2023-11-24T06:16:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.955+0000 7f59cbfff640 1 -- 172.21.15.5:0/2965930749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4014b30 con 0x7f59d40b3040 2023-11-24T06:16:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.955+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2965930749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d40b6670 con 0x7f59d40b3040 2023-11-24T06:16:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.956+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2965930749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d40b6830 con 0x7f59d40b3040 2023-11-24T06:16:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.957+0000 7f59cbfff640 1 -- 172.21.15.5:0/2965930749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f59c4014c90 con 0x7f59d40b3040 2023-11-24T06:16:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.958+0000 7f59cbfff640 1 --2- 172.21.15.5:0/2965930749 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f59b406ae70 0x7f59b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.958+0000 7f59cbfff640 1 -- 172.21.15.5:0/2965930749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59c4089fd0 con 0x7f59d40b3040 2023-11-24T06:16:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.958+0000 7f59d2ffd640 1 --2- 172.21.15.5:0/2965930749 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f59b406ae70 0x7f59b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.959+0000 7f59d2ffd640 1 --2- 172.21.15.5:0/2965930749 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f59b406ae70 0x7f59b406d330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f59d40d16e0 tx=0x7f59bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:27.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:27.962+0000 7f59cbfff640 1 -- 172.21.15.5:0/2965930749 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f59c4054ff0 con 0x7f59d40b3040 2023-11-24T06:16:28.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.153+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2965930749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59d40a1210 con 0x7f59d40b3040 2023-11-24T06:16:28.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.154+0000 7f59cbfff640 1 -- 172.21.15.5:0/2965930749 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f59c4058610 con 0x7f59d40b3040 2023-11-24T06:16:28.156 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:28.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.157+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2965930749 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f59b406ae70 msgr2=0x7f59b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:28.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.157+0000 7f59d9cbb640 1 --2- 172.21.15.5:0/2965930749 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f59b406ae70 0x7f59b406d330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f59d40d16e0 tx=0x7f59bc008040 comp rx=0 tx=0).stop 2023-11-24T06:16:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.157+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2965930749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d40b3040 msgr2=0x7f59d41543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.157+0000 7f59d9cbb640 1 --2- 172.21.15.5:0/2965930749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d40b3040 0x7f59d41543d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f59c4007e70 tx=0x7f59c400e4b0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.157+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2965930749 shutdown_connections 2023-11-24T06:16:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.158+0000 7f59d9cbb640 1 --2- 172.21.15.5:0/2965930749 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f59b406ae70 0x7f59b406d330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.158+0000 7f59d9cbb640 1 --2- 172.21.15.5:0/2965930749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f59d40b48d0 0x7f59d41526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.158+0000 7f59d9cbb640 1 --2- 172.21.15.5:0/2965930749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f59d4160bd0 0x7f59d40c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.158+0000 7f59d9cbb640 1 --2- 172.21.15.5:0/2965930749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59d40b3040 0x7f59d41543d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.158+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2965930749 >> 172.21.15.5:0/2965930749 conn(0x7f59d409c5a0 msgr2=0x7f59d40d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.158+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2965930749 shutdown_connections 2023-11-24T06:16:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.158+0000 7f59d9cbb640 1 -- 172.21.15.5:0/2965930749 wait complete. 2023-11-24T06:16:28.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:28.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.387+0000 7fbf6247e640 1 Processor -- start 2023-11-24T06:16:28.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.387+0000 7fbf6247e640 1 -- start start 2023-11-24T06:16:28.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.389+0000 7fbf6247e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fbf5c1435d0 0x7fbf5c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:28.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.389+0000 7fbf6247e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf5c1439b0 0x7fbf5c0b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:28.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.389+0000 7fbf6247e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5c070d40 con 0x7fbf5c0b3f70 2023-11-24T06:16:28.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.389+0000 7fbf6247e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5c0b4400 con 0x7fbf5c1435d0 2023-11-24T06:16:28.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.389+0000 7fbf6247e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c071020 con 0x7fbf5c1439b0 2023-11-24T06:16:28.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.389+0000 7fbf61c7d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbf5c0b3f70 0x7fbf5c070050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35498/0 (socket says 172.21.15.5:35498) 2023-11-24T06:16:28.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.389+0000 7fbf61c7d640 1 -- 172.21.15.5:0/3483820875 learned_addr learned my addr 172.21.15.5:0/3483820875 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.389+0000 7fbf6147c640 1 --2- 172.21.15.5:0/3483820875 >> v2:172.21.15.6:3300/0 conn(0x7fbf5c1435d0 0x7fbf5c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.389+0000 7fbf60c7b640 1 --2- 172.21.15.5:0/3483820875 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf5c1439b0 0x7fbf5c0b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:28.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 -- 172.21.15.5:0/3483820875 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716627140 0 0) 0x7fbf5c070d40 con 0x7fbf5c0b3f70 2023-11-24T06:16:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 -- 172.21.15.5:0/3483820875 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf48003520 con 0x7fbf5c0b3f70 2023-11-24T06:16:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 -- 172.21.15.5:0/3483820875 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1189867006 0 0) 0x7fbf48003520 con 0x7fbf5c0b3f70 2023-11-24T06:16:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 -- 172.21.15.5:0/3483820875 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf5c1439b0 msgr2=0x7fbf5c0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 --2- 172.21.15.5:0/3483820875 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf5c1439b0 0x7fbf5c0b3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 -- 172.21.15.5:0/3483820875 >> v2:172.21.15.6:3300/0 conn(0x7fbf5c1435d0 msgr2=0x7fbf5c071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 --2- 172.21.15.5:0/3483820875 >> v2:172.21.15.6:3300/0 conn(0x7fbf5c1435d0 0x7fbf5c071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 -- 172.21.15.5:0/3483820875 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c071300 con 0x7fbf5c0b3f70 2023-11-24T06:16:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 -- 172.21.15.5:0/3483820875 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fbf540019f0 con 0x7fbf5c0b3f70 2023-11-24T06:16:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 -- 172.21.15.5:0/3483820875 >> v1:172.21.15.5:6789/0 conn(0x7fbf5c0b3f70 legacy=0x7fbf5c070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 --2- 172.21.15.5:0/3483820875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf480039e0 0x7fbf48003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:28.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 --2- 172.21.15.5:0/3483820875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf480047c0 0x7fbf48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 --2- 172.21.15.5:0/3483820875 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf5c1439b0 0x7fbf48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 -- 172.21.15.5:0/3483820875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0b4400 con 0x7fbf480039e0 2023-11-24T06:16:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 -- 172.21.15.5:0/3483820875 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c071020 con 0x7fbf480047c0 2023-11-24T06:16:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.390+0000 7fbf53fff640 1 -- 172.21.15.5:0/3483820875 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4800dbc0 con 0x7fbf5c1439b0 2023-11-24T06:16:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.391+0000 7fbf6147c640 1 --2- 172.21.15.5:0/3483820875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf480039e0 0x7fbf48003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.391+0000 7fbf61c7d640 1 --2- 172.21.15.5:0/3483820875 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf5c1439b0 0x7fbf48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.391+0000 7fbf6147c640 1 -- 172.21.15.5:0/3483820875 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf5c1439b0 msgr2=0x7fbf48005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.391+0000 7fbf6147c640 1 --2- 172.21.15.5:0/3483820875 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf5c1439b0 0x7fbf48005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.391+0000 7fbf6147c640 1 -- 172.21.15.5:0/3483820875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf480047c0 msgr2=0x7fbf48004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.391+0000 7fbf6147c640 1 --2- 172.21.15.5:0/3483820875 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf480047c0 0x7fbf48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.391+0000 7fbf6147c640 1 -- 172.21.15.5:0/3483820875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf48003520 con 0x7fbf480039e0 2023-11-24T06:16:28.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.392+0000 7fbf6147c640 1 --2- 172.21.15.5:0/3483820875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf480039e0 0x7fbf48003e00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009ac0 tx=0x7fbf480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a820eb9374c1ca server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:28.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.393+0000 7fbf53fff640 1 -- 172.21.15.5:0/3483820875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5802c6d0 con 0x7fbf480039e0 2023-11-24T06:16:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.393+0000 7fbf53fff640 1 -- 172.21.15.5:0/3483820875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5802c830 con 0x7fbf480039e0 2023-11-24T06:16:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.393+0000 7fbf53fff640 1 -- 172.21.15.5:0/3483820875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5802cbc0 con 0x7fbf480039e0 2023-11-24T06:16:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.393+0000 7fbf6247e640 1 -- 172.21.15.5:0/3483820875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf480039e0 msgr2=0x7fbf48003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.393+0000 7fbf6247e640 1 --2- 172.21.15.5:0/3483820875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf480039e0 0x7fbf48003e00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf58009ac0 tx=0x7fbf480057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.393+0000 7fbf61c7d640 1 -- 172.21.15.5:0/3483820875 reap_dead start 2023-11-24T06:16:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.393+0000 7fbf6247e640 1 -- 172.21.15.5:0/3483820875 shutdown_connections 2023-11-24T06:16:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.393+0000 7fbf6247e640 1 -- 172.21.15.5:0/3483820875 >> 172.21.15.5:0/3483820875 conn(0x7fbf5c0cec10 msgr2=0x7fbf5c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.393+0000 7fbf6247e640 1 -- 172.21.15.5:0/3483820875 shutdown_connections 2023-11-24T06:16:28.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.393+0000 7fbf6247e640 1 -- 172.21.15.5:0/3483820875 wait complete. 2023-11-24T06:16:28.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.396+0000 7fbf6247e640 1 Processor -- start 2023-11-24T06:16:28.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.396+0000 7fbf6247e640 1 -- start start 2023-11-24T06:16:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf6247e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf480039e0 0x7fbf5c159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf6247e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf5c159630 0x7fbf5c13d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf6247e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0ae930 0x7fbf5c0aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf6247e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c140820 con 0x7fbf5c0ae930 2023-11-24T06:16:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf6247e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0af2f0 con 0x7fbf480039e0 2023-11-24T06:16:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf6247e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c13d980 con 0x7fbf5c159630 2023-11-24T06:16:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf61c7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0ae930 0x7fbf5c0aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf61c7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0ae930 0x7fbf5c0aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40438/0 (socket says 172.21.15.5:40438) 2023-11-24T06:16:28.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf61c7d640 1 -- 172.21.15.5:0/1506967221 learned_addr learned my addr 172.21.15.5:0/1506967221 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf6147c640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf480039e0 0x7fbf5c159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf60c7b640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf5c159630 0x7fbf5c13d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf61c7d640 1 -- 172.21.15.5:0/1506967221 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf5c159630 msgr2=0x7fbf5c13d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf61c7d640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf5c159630 0x7fbf5c13d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf61c7d640 1 -- 172.21.15.5:0/1506967221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf480039e0 msgr2=0x7fbf5c159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf61c7d640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf480039e0 0x7fbf5c159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf61c7d640 1 -- 172.21.15.5:0/1506967221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c13cfb0 con 0x7fbf5c0ae930 2023-11-24T06:16:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.397+0000 7fbf6147c640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf480039e0 0x7fbf5c159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.398+0000 7fbf61c7d640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0ae930 0x7fbf5c0aedb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbf540012e0 tx=0x7fbf5400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.398+0000 7fbf51ffb640 1 -- 172.21.15.5:0/1506967221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf54004d60 con 0x7fbf5c0ae930 2023-11-24T06:16:28.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.398+0000 7fbf51ffb640 1 -- 172.21.15.5:0/1506967221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5400bb20 con 0x7fbf5c0ae930 2023-11-24T06:16:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.398+0000 7fbf51ffb640 1 -- 172.21.15.5:0/1506967221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400bed0 con 0x7fbf5c0ae930 2023-11-24T06:16:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.398+0000 7fbf6247e640 1 -- 172.21.15.5:0/1506967221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5c13c680 con 0x7fbf5c0ae930 2023-11-24T06:16:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.398+0000 7fbf6247e640 1 -- 172.21.15.5:0/1506967221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5c13cc40 con 0x7fbf5c0ae930 2023-11-24T06:16:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.400+0000 7fbf6247e640 1 -- 172.21.15.5:0/1506967221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5c13d140 con 0x7fbf5c0ae930 2023-11-24T06:16:28.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.400+0000 7fbf51ffb640 1 -- 172.21.15.5:0/1506967221 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbf54007280 con 0x7fbf5c0ae930 2023-11-24T06:16:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.401+0000 7fbf51ffb640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbf4006ae70 0x7fbf4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:28.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.402+0000 7fbf51ffb640 1 -- 172.21.15.5:0/1506967221 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf54089ba0 con 0x7fbf5c0ae930 2023-11-24T06:16:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.402+0000 7fbf6147c640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbf4006ae70 0x7fbf4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:28.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.403+0000 7fbf6147c640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbf4006ae70 0x7fbf4006d330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf58036000 tx=0x7fbf58030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:28.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.406+0000 7fbf51ffb640 1 -- 172.21.15.5:0/1506967221 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf54054bc0 con 0x7fbf5c0ae930 2023-11-24T06:16:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.598+0000 7fbf6247e640 1 -- 172.21.15.5:0/1506967221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf5c04e840 con 0x7fbf5c0ae930 2023-11-24T06:16:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.599+0000 7fbf51ffb640 1 -- 172.21.15.5:0/1506967221 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbf540581e0 con 0x7fbf5c0ae930 2023-11-24T06:16:28.602 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.601+0000 7fbf6247e640 1 -- 172.21.15.5:0/1506967221 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbf4006ae70 msgr2=0x7fbf4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.601+0000 7fbf6247e640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbf4006ae70 0x7fbf4006d330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf58036000 tx=0x7fbf58030000 comp rx=0 tx=0).stop 2023-11-24T06:16:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.602+0000 7fbf6247e640 1 -- 172.21.15.5:0/1506967221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0ae930 msgr2=0x7fbf5c0aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.602+0000 7fbf6247e640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0ae930 0x7fbf5c0aedb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbf540012e0 tx=0x7fbf5400b4a0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.602+0000 7fbf6247e640 1 -- 172.21.15.5:0/1506967221 shutdown_connections 2023-11-24T06:16:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.602+0000 7fbf6247e640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbf4006ae70 0x7fbf4006d330 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.602+0000 7fbf6247e640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbf5c159630 0x7fbf5c13d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.602+0000 7fbf6247e640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5c0ae930 0x7fbf5c0aedb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.602+0000 7fbf6247e640 1 --2- 172.21.15.5:0/1506967221 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbf480039e0 0x7fbf5c159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.602+0000 7fbf6247e640 1 -- 172.21.15.5:0/1506967221 >> 172.21.15.5:0/1506967221 conn(0x7fbf5c0cec10 msgr2=0x7fbf5c06cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.602+0000 7fbf6247e640 1 -- 172.21.15.5:0/1506967221 shutdown_connections 2023-11-24T06:16:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.602+0000 7fbf6247e640 1 -- 172.21.15.5:0/1506967221 wait complete. 2023-11-24T06:16:28.623 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.843+0000 7fbea2747640 1 Processor -- start 2023-11-24T06:16:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.843+0000 7fbea2747640 1 -- start start 2023-11-24T06:16:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.845+0000 7fbea2747640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbe9c0c8930 0x7fbe9c0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:28.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.845+0000 7fbea2747640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9c0c6bf0 con 0x7fbe9c15d3b0 2023-11-24T06:16:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.845+0000 7fbea2747640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9c0cedc0 con 0x7fbe9c0d1d90 2023-11-24T06:16:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.845+0000 7fbea2747640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9c0c6e40 con 0x7fbe9c0c8930 2023-11-24T06:16:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.845+0000 7fbea1745640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbe9c15d3b0 0x7fbe9c0d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35506/0 (socket says 172.21.15.5:35506) 2023-11-24T06:16:28.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.845+0000 7fbea1745640 1 -- 172.21.15.5:0/2530306356 learned_addr learned my addr 172.21.15.5:0/2530306356 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.845+0000 7fbea1f46640 1 --2- 172.21.15.5:0/2530306356 >> v2:172.21.15.5:3300/0 conn(0x7fbe9c0c8930 0x7fbe9c0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.846+0000 7fbe93fff640 1 -- 172.21.15.5:0/2530306356 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276110862 0 0) 0x7fbe9c0c6bf0 con 0x7fbe9c15d3b0 2023-11-24T06:16:28.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.846+0000 7fbe93fff640 1 -- 172.21.15.5:0/2530306356 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe88003520 con 0x7fbe9c15d3b0 2023-11-24T06:16:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.846+0000 7fbea1f46640 1 -- 172.21.15.5:0/2530306356 >> v1:172.21.15.6:6789/0 conn(0x7fbe9c0d1d90 legacy=0x7fbe9c0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.846+0000 7fbea1f46640 1 -- 172.21.15.5:0/2530306356 >> v1:172.21.15.5:6789/0 conn(0x7fbe9c15d3b0 legacy=0x7fbe9c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.846+0000 7fbea1f46640 1 -- 172.21.15.5:0/2530306356 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9c0c7120 con 0x7fbe9c0c8930 2023-11-24T06:16:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.847+0000 7fbea1f46640 1 --2- 172.21.15.5:0/2530306356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9c0c8930 0x7fbe9c0ce6c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009d90 tx=0x7fbe9402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa306e491aaa315 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.848+0000 7fbe93fff640 1 -- 172.21.15.5:0/2530306356 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9402cc60 con 0x7fbe9c0c8930 2023-11-24T06:16:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.848+0000 7fbe93fff640 1 -- 172.21.15.5:0/2530306356 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9402cdc0 con 0x7fbe9c0c8930 2023-11-24T06:16:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.848+0000 7fbe93fff640 1 -- 172.21.15.5:0/2530306356 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94034660 con 0x7fbe9c0c8930 2023-11-24T06:16:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.848+0000 7fbea2747640 1 -- 172.21.15.5:0/2530306356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9c0c8930 msgr2=0x7fbe9c0ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:28.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.848+0000 7fbea2747640 1 --2- 172.21.15.5:0/2530306356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9c0c8930 0x7fbe9c0ce6c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009d90 tx=0x7fbe9402beb0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.848+0000 7fbea2747640 1 -- 172.21.15.5:0/2530306356 shutdown_connections 2023-11-24T06:16:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.848+0000 7fbea2747640 1 --2- 172.21.15.5:0/2530306356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9c0c8930 0x7fbe9c0ce6c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.848+0000 7fbea2747640 1 -- 172.21.15.5:0/2530306356 >> 172.21.15.5:0/2530306356 conn(0x7fbe9c0d30a0 msgr2=0x7fbe9c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.849+0000 7fbea2747640 1 -- 172.21.15.5:0/2530306356 shutdown_connections 2023-11-24T06:16:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.849+0000 7fbea2747640 1 -- 172.21.15.5:0/2530306356 wait complete. 2023-11-24T06:16:28.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.850+0000 7fbea2747640 1 Processor -- start 2023-11-24T06:16:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.850+0000 7fbea2747640 1 -- start start 2023-11-24T06:16:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.851+0000 7fbea2747640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe9c0c8930 0x7fbe9c0c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.851+0000 7fbea2747640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe9c0d1d90 0x7fbe9c158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.851+0000 7fbea2747640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9c15d3b0 0x7fbe9c15a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.851+0000 7fbea2747640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c0b8c20 con 0x7fbe9c15d3b0 2023-11-24T06:16:28.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.851+0000 7fbea2747640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c158870 con 0x7fbe9c0d1d90 2023-11-24T06:16:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.851+0000 7fbea2747640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9c15aca0 con 0x7fbe9c0c8930 2023-11-24T06:16:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.851+0000 7fbea1f46640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9c15d3b0 0x7fbe9c15a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.852+0000 7fbea1f46640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9c15d3b0 0x7fbe9c15a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40458/0 (socket says 172.21.15.5:40458) 2023-11-24T06:16:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.852+0000 7fbea1f46640 1 -- 172.21.15.5:0/2813754585 learned_addr learned my addr 172.21.15.5:0/2813754585 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.852+0000 7fbea1745640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe9c0c8930 0x7fbe9c0c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:28.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.852+0000 7fbea1f46640 1 -- 172.21.15.5:0/2813754585 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe9c0c8930 msgr2=0x7fbe9c0c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.852+0000 7fbea1f46640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe9c0c8930 0x7fbe9c0c6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.852+0000 7fbea1f46640 1 -- 172.21.15.5:0/2813754585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe9c0d1d90 msgr2=0x7fbe9c158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.852+0000 7fbea0f44640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe9c0d1d90 0x7fbe9c158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.852+0000 7fbea1f46640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe9c0d1d90 0x7fbe9c158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.852+0000 7fbea1f46640 1 -- 172.21.15.5:0/2813754585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe94009a70 con 0x7fbe9c15d3b0 2023-11-24T06:16:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.852+0000 7fbea0f44640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe9c0d1d90 0x7fbe9c158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:28.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.852+0000 7fbea1f46640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9c15d3b0 0x7fbe9c15a760 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009eb0 tx=0x7fbe94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:28.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.853+0000 7fbe91ffb640 1 -- 172.21.15.5:0/2813754585 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9403e610 con 0x7fbe9c15d3b0 2023-11-24T06:16:28.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.853+0000 7fbe91ffb640 1 -- 172.21.15.5:0/2813754585 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9403e770 con 0x7fbe9c15d3b0 2023-11-24T06:16:28.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.853+0000 7fbe91ffb640 1 -- 172.21.15.5:0/2813754585 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9403eb20 con 0x7fbe9c15d3b0 2023-11-24T06:16:28.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.853+0000 7fbea2747640 1 -- 172.21.15.5:0/2813754585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9c15b1e0 con 0x7fbe9c15d3b0 2023-11-24T06:16:28.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.853+0000 7fbea2747640 1 -- 172.21.15.5:0/2813754585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9c0cb2d0 con 0x7fbe9c15d3b0 2023-11-24T06:16:28.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.854+0000 7fbea2747640 1 -- 172.21.15.5:0/2813754585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe9c04e840 con 0x7fbe9c15d3b0 2023-11-24T06:16:28.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.855+0000 7fbe91ffb640 1 -- 172.21.15.5:0/2813754585 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbe9403b050 con 0x7fbe9c15d3b0 2023-11-24T06:16:28.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.856+0000 7fbe91ffb640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbe8006ae90 0x7fbe8006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:28.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.856+0000 7fbe91ffb640 1 -- 172.21.15.5:0/2813754585 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe940abe20 con 0x7fbe9c15d3b0 2023-11-24T06:16:28.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.856+0000 7fbea1745640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbe8006ae90 0x7fbe8006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:28.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.857+0000 7fbea1745640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbe8006ae90 0x7fbe8006d350 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbe980011e0 tx=0x7fbe98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:28.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:28.860+0000 7fbe91ffb640 1 -- 172.21.15.5:0/2813754585 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbe94076f40 con 0x7fbe9c15d3b0 2023-11-24T06:16:29.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.049+0000 7fbea2747640 1 -- 172.21.15.5:0/2813754585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe9c0cdfe0 con 0x7fbe9c15d3b0 2023-11-24T06:16:29.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.050+0000 7fbe91ffb640 1 -- 172.21.15.5:0/2813754585 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe9407a560 con 0x7fbe9c15d3b0 2023-11-24T06:16:29.054 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.052+0000 7fbea2747640 1 -- 172.21.15.5:0/2813754585 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbe8006ae90 msgr2=0x7fbe8006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.052+0000 7fbea2747640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbe8006ae90 0x7fbe8006d350 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbe980011e0 tx=0x7fbe98009040 comp rx=0 tx=0).stop 2023-11-24T06:16:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.052+0000 7fbea2747640 1 -- 172.21.15.5:0/2813754585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9c15d3b0 msgr2=0x7fbe9c15a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:29.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.052+0000 7fbea2747640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9c15d3b0 0x7fbe9c15a760 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009eb0 tx=0x7fbe94004a40 comp rx=0 tx=0).stop 2023-11-24T06:16:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.052+0000 7fbea2747640 1 -- 172.21.15.5:0/2813754585 shutdown_connections 2023-11-24T06:16:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.052+0000 7fbea2747640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fbe8006ae90 0x7fbe8006d350 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.052+0000 7fbea2747640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9c15d3b0 0x7fbe9c15a760 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.052+0000 7fbea2747640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fbe9c0d1d90 0x7fbe9c158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.052+0000 7fbea2747640 1 --2- 172.21.15.5:0/2813754585 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fbe9c0c8930 0x7fbe9c0c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.052+0000 7fbea2747640 1 -- 172.21.15.5:0/2813754585 >> 172.21.15.5:0/2813754585 conn(0x7fbe9c0d30a0 msgr2=0x7fbe9c0b5490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:29.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.053+0000 7fbea2747640 1 -- 172.21.15.5:0/2813754585 shutdown_connections 2023-11-24T06:16:29.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.053+0000 7fbea2747640 1 -- 172.21.15.5:0/2813754585 wait complete. 2023-11-24T06:16:29.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.299+0000 7f0d1dbba640 1 Processor -- start 2023-11-24T06:16:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.299+0000 7f0d1dbba640 1 -- start start 2023-11-24T06:16:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.301+0000 7f0d1dbba640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d1815d5b0 0x7f0d180d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.301+0000 7f0d1dbba640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0d180b1650 0x7f0d180d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.301+0000 7f0d1dbba640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d180cf0c0 con 0x7f0d180d14a0 2023-11-24T06:16:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.301+0000 7f0d1dbba640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d180b01b0 con 0x7f0d180b1650 2023-11-24T06:16:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.301+0000 7f0d1dbba640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1815d990 con 0x7f0d1815d5b0 2023-11-24T06:16:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.302+0000 7f0d1d3b9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0d180b1650 0x7f0d180d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.302+0000 7f0d1cbb8640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d1815d5b0 0x7f0d180d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.302+0000 7f0d1d3b9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0d180b1650 0x7f0d180d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:37600/0 (socket says 172.21.15.5:37600) 2023-11-24T06:16:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.302+0000 7f0d1d3b9640 1 -- 172.21.15.5:0/713834737 learned_addr learned my addr 172.21.15.5:0/713834737 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.302+0000 7f0d1d3b9640 1 -- 172.21.15.5:0/713834737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d1815d5b0 msgr2=0x7f0d180d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.302+0000 7f0d1d3b9640 1 --2- 172.21.15.5:0/713834737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d1815d5b0 0x7f0d180d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.302+0000 7f0d1d3b9640 1 -- 172.21.15.5:0/713834737 >> v1:172.21.15.6:6789/0 conn(0x7f0d180d14a0 legacy=0x7f0d180cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.302+0000 7f0d1d3b9640 1 -- 172.21.15.5:0/713834737 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d180d1e40 con 0x7f0d180b1650 2023-11-24T06:16:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.304+0000 7f0d1d3b9640 1 --2- 172.21.15.5:0/713834737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d180b1650 0x7f0d180d1900 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009e50 tx=0x7f0d1002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c891892d73474de0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.305+0000 7f0d177fe640 1 -- 172.21.15.5:0/713834737 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1002cb00 con 0x7f0d180b1650 2023-11-24T06:16:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.305+0000 7f0d177fe640 1 -- 172.21.15.5:0/713834737 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d1002cc60 con 0x7f0d180b1650 2023-11-24T06:16:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.305+0000 7f0d177fe640 1 -- 172.21.15.5:0/713834737 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d100343f0 con 0x7f0d180b1650 2023-11-24T06:16:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.305+0000 7f0d1dbba640 1 -- 172.21.15.5:0/713834737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d180b1650 msgr2=0x7f0d180d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:29.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.305+0000 7f0d1dbba640 1 --2- 172.21.15.5:0/713834737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d180b1650 0x7f0d180d1900 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009e50 tx=0x7f0d1002bf60 comp rx=0 tx=0).stop 2023-11-24T06:16:29.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.305+0000 7f0d1dbba640 1 -- 172.21.15.5:0/713834737 shutdown_connections 2023-11-24T06:16:29.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.305+0000 7f0d1dbba640 1 --2- 172.21.15.5:0/713834737 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d180b1650 0x7f0d180d1900 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.305+0000 7f0d1dbba640 1 --2- 172.21.15.5:0/713834737 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d1815d5b0 0x7f0d180d3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.305+0000 7f0d1dbba640 1 -- 172.21.15.5:0/713834737 >> 172.21.15.5:0/713834737 conn(0x7f0d1809c3b0 msgr2=0x7f0d180d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:29.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.305+0000 7f0d1dbba640 1 -- 172.21.15.5:0/713834737 shutdown_connections 2023-11-24T06:16:29.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.305+0000 7f0d1dbba640 1 -- 172.21.15.5:0/713834737 wait complete. 2023-11-24T06:16:29.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.307+0000 7f0d1dbba640 1 Processor -- start 2023-11-24T06:16:29.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.307+0000 7f0d1dbba640 1 -- start start 2023-11-24T06:16:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.308+0000 7f0d1dbba640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d180b1650 0x7f0d180b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.308+0000 7f0d1dbba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d180d14a0 0x7f0d18152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.309+0000 7f0d1dbba640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d1815d5b0 0x7f0d180cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.309+0000 7f0d1dbba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1815e4d0 con 0x7f0d180d14a0 2023-11-24T06:16:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.309+0000 7f0d1dbba640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1815e660 con 0x7f0d1815d5b0 2023-11-24T06:16:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.309+0000 7f0d1dbba640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181529d0 con 0x7f0d180b1650 2023-11-24T06:16:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.309+0000 7f0d17fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d180d14a0 0x7f0d18152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.309+0000 7f0d17fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d180d14a0 0x7f0d18152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40460/0 (socket says 172.21.15.5:40460) 2023-11-24T06:16:29.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.309+0000 7f0d17fff640 1 -- 172.21.15.5:0/1860616981 learned_addr learned my addr 172.21.15.5:0/1860616981 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:29.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.309+0000 7f0d1cbb8640 1 --2- 172.21.15.5:0/1860616981 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d180b1650 0x7f0d180b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.309+0000 7f0d1d3b9640 1 --2- 172.21.15.5:0/1860616981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d1815d5b0 0x7f0d180cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.309+0000 7f0d17fff640 1 -- 172.21.15.5:0/1860616981 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d180b1650 msgr2=0x7f0d180b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.309+0000 7f0d17fff640 1 --2- 172.21.15.5:0/1860616981 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d180b1650 0x7f0d180b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.309+0000 7f0d17fff640 1 -- 172.21.15.5:0/1860616981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d1815d5b0 msgr2=0x7f0d180cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.309+0000 7f0d17fff640 1 --2- 172.21.15.5:0/1860616981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d1815d5b0 0x7f0d180cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.309+0000 7f0d17fff640 1 -- 172.21.15.5:0/1860616981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d10009b30 con 0x7f0d180d14a0 2023-11-24T06:16:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.310+0000 7f0d17fff640 1 --2- 172.21.15.5:0/1860616981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d180d14a0 0x7f0d18152490 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400c400 tx=0x7f0d0400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.310+0000 7f0d157fa640 1 -- 172.21.15.5:0/1860616981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d040052d0 con 0x7f0d180d14a0 2023-11-24T06:16:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.310+0000 7f0d157fa640 1 -- 172.21.15.5:0/1860616981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d04005430 con 0x7f0d180d14a0 2023-11-24T06:16:29.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.310+0000 7f0d1dbba640 1 -- 172.21.15.5:0/1860616981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d180b6cb0 con 0x7f0d180d14a0 2023-11-24T06:16:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.310+0000 7f0d157fa640 1 -- 172.21.15.5:0/1860616981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04003830 con 0x7f0d180d14a0 2023-11-24T06:16:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.310+0000 7f0d1dbba640 1 -- 172.21.15.5:0/1860616981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d180b7040 con 0x7f0d180d14a0 2023-11-24T06:16:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.311+0000 7f0d1dbba640 1 -- 172.21.15.5:0/1860616981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1804e840 con 0x7f0d180d14a0 2023-11-24T06:16:29.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.312+0000 7f0d157fa640 1 -- 172.21.15.5:0/1860616981 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0d04003990 con 0x7f0d180d14a0 2023-11-24T06:16:29.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.313+0000 7f0d157fa640 1 --2- 172.21.15.5:0/1860616981 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0cfc06ae40 0x7f0cfc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.313+0000 7f0d157fa640 1 -- 172.21.15.5:0/1860616981 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d04088c10 con 0x7f0d180d14a0 2023-11-24T06:16:29.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.314+0000 7f0d1cbb8640 1 --2- 172.21.15.5:0/1860616981 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0cfc06ae40 0x7f0cfc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.314+0000 7f0d1cbb8640 1 --2- 172.21.15.5:0/1860616981 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0cfc06ae40 0x7f0cfc06d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009a30 tx=0x7f0d0c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.317+0000 7f0d157fa640 1 -- 172.21.15.5:0/1860616981 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d04053c20 con 0x7f0d180d14a0 2023-11-24T06:16:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.503+0000 7f0d1dbba640 1 -- 172.21.15.5:0/1860616981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d180678d0 con 0x7f0d180d14a0 2023-11-24T06:16:29.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.506+0000 7f0d157fa640 1 -- 172.21.15.5:0/1860616981 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16361 (secure 0 0 0) 0x7f0d04017090 con 0x7f0d180d14a0 2023-11-24T06:16:29.510 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:29.510 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:52.779059+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:29.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.509+0000 7f0d1dbba640 1 -- 172.21.15.5:0/1860616981 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0cfc06ae40 msgr2=0x7f0cfc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:29.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.509+0000 7f0d1dbba640 1 --2- 172.21.15.5:0/1860616981 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0cfc06ae40 0x7f0cfc06d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009a30 tx=0x7f0d0c008040 comp rx=0 tx=0).stop 2023-11-24T06:16:29.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.509+0000 7f0d1dbba640 1 -- 172.21.15.5:0/1860616981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d180d14a0 msgr2=0x7f0d18152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:29.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.510+0000 7f0d1dbba640 1 --2- 172.21.15.5:0/1860616981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d180d14a0 0x7f0d18152490 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400c400 tx=0x7f0d0400c8c0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.510+0000 7f0d1dbba640 1 -- 172.21.15.5:0/1860616981 shutdown_connections 2023-11-24T06:16:29.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.510+0000 7f0d1dbba640 1 --2- 172.21.15.5:0/1860616981 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0cfc06ae40 0x7f0cfc06d300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.510+0000 7f0d1dbba640 1 --2- 172.21.15.5:0/1860616981 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0d1815d5b0 0x7f0d180cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.510+0000 7f0d1dbba640 1 --2- 172.21.15.5:0/1860616981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d180d14a0 0x7f0d18152490 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.510+0000 7f0d1dbba640 1 --2- 172.21.15.5:0/1860616981 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0d180b1650 0x7f0d180b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.510+0000 7f0d1dbba640 1 -- 172.21.15.5:0/1860616981 >> 172.21.15.5:0/1860616981 conn(0x7f0d1809c3b0 msgr2=0x7f0d180b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:29.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.510+0000 7f0d1dbba640 1 -- 172.21.15.5:0/1860616981 shutdown_connections 2023-11-24T06:16:29.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.510+0000 7f0d1dbba640 1 -- 172.21.15.5:0/1860616981 wait complete. 2023-11-24T06:16:29.532 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:29.533 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:29.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.764+0000 7ff8588ee640 1 Processor -- start 2023-11-24T06:16:29.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.764+0000 7ff8588ee640 1 -- start start 2023-11-24T06:16:29.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.766+0000 7ff8588ee640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff854143050 0x7ff85415e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.766+0000 7ff8588ee640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8541435d0 0x7ff8540b4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.766+0000 7ff8588ee640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8540b3c00 con 0x7ff854143b20 2023-11-24T06:16:29.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.766+0000 7ff8588ee640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85415e510 con 0x7ff854143050 2023-11-24T06:16:29.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.766+0000 7ff8588ee640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff854143430 con 0x7ff8541435d0 2023-11-24T06:16:29.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.767+0000 7ff852d9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff854143b20 0x7ff85415a7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35516/0 (socket says 172.21.15.5:35516) 2023-11-24T06:16:29.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.767+0000 7ff852d9d640 1 -- 172.21.15.5:0/3075715471 learned_addr learned my addr 172.21.15.5:0/3075715471 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.767+0000 7ff85259c640 1 --2- 172.21.15.5:0/3075715471 >> v2:172.21.15.6:3300/0 conn(0x7ff854143050 0x7ff85415e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.767+0000 7ff849d9b640 1 --2- 172.21.15.5:0/3075715471 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8541435d0 0x7ff8540b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.767+0000 7ff851d9b640 1 -- 172.21.15.5:0/3075715471 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666809292 0 0) 0x7ff8540b3c00 con 0x7ff854143b20 2023-11-24T06:16:29.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.767+0000 7ff851d9b640 1 -- 172.21.15.5:0/3075715471 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff840003520 con 0x7ff854143b20 2023-11-24T06:16:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff851d9b640 1 -- 172.21.15.5:0/3075715471 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3477120837 0 0) 0x7ff840003520 con 0x7ff854143b20 2023-11-24T06:16:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff851d9b640 1 -- 172.21.15.5:0/3075715471 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8541435d0 msgr2=0x7ff8540b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff851d9b640 1 --2- 172.21.15.5:0/3075715471 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8541435d0 0x7ff8540b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff851d9b640 1 -- 172.21.15.5:0/3075715471 >> v2:172.21.15.6:3300/0 conn(0x7ff854143050 msgr2=0x7ff85415e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:29.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff851d9b640 1 --2- 172.21.15.5:0/3075715471 >> v2:172.21.15.6:3300/0 conn(0x7ff854143050 0x7ff85415e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff851d9b640 1 -- 172.21.15.5:0/3075715471 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85415de50 con 0x7ff854143b20 2023-11-24T06:16:29.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff851d9b640 1 -- 172.21.15.5:0/3075715471 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7ff84c0019f0 con 0x7ff854143b20 2023-11-24T06:16:29.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff851d9b640 1 -- 172.21.15.5:0/3075715471 >> v1:172.21.15.5:6789/0 conn(0x7ff854143b20 legacy=0x7ff85415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:29.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff851d9b640 1 --2- 172.21.15.5:0/3075715471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8400039a0 0x7ff840003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff851d9b640 1 --2- 172.21.15.5:0/3075715471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8400047c0 0x7ff840004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff851d9b640 1 --2- 172.21.15.5:0/3075715471 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff840005180 0x7ff84000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff851d9b640 1 -- 172.21.15.5:0/3075715471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85415e510 con 0x7ff8400047c0 2023-11-24T06:16:29.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff851d9b640 1 -- 172.21.15.5:0/3075715471 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff854143430 con 0x7ff8400039a0 2023-11-24T06:16:29.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff851d9b640 1 -- 172.21.15.5:0/3075715471 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff84000e150 con 0x7ff840005180 2023-11-24T06:16:29.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff849d9b640 1 --2- 172.21.15.5:0/3075715471 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8541435d0 0x7ff8540b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:16:29.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.768+0000 7ff849d9b640 1 --2- 172.21.15.5:0/3075715471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8400047c0 0x7ff840004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.769+0000 7ff85259c640 1 --2- 172.21.15.5:0/3075715471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8400039a0 0x7ff840003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.769+0000 7ff849d9b640 1 -- 172.21.15.5:0/3075715471 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff840005180 msgr2=0x7ff84000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.769+0000 7ff852d9d640 1 --2- 172.21.15.5:0/3075715471 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff840005180 0x7ff84000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.769+0000 7ff849d9b640 1 --2- 172.21.15.5:0/3075715471 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff840005180 0x7ff84000db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.769+0000 7ff849d9b640 1 -- 172.21.15.5:0/3075715471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8400039a0 msgr2=0x7ff840003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:29.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.769+0000 7ff849d9b640 1 --2- 172.21.15.5:0/3075715471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8400039a0 0x7ff840003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.769+0000 7ff849d9b640 1 -- 172.21.15.5:0/3075715471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff840003520 con 0x7ff8400047c0 2023-11-24T06:16:29.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.769+0000 7ff85259c640 1 --2- 172.21.15.5:0/3075715471 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff8400039a0 0x7ff840003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:29.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.770+0000 7ff849d9b640 1 --2- 172.21.15.5:0/3075715471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8400047c0 0x7ff840004c40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff854144190 tx=0x7ff8400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc2b36b101a6fe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:29.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.770+0000 7ff851d9b640 1 -- 172.21.15.5:0/3075715471 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c004030 con 0x7ff8400047c0 2023-11-24T06:16:29.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.770+0000 7ff851d9b640 1 -- 172.21.15.5:0/3075715471 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff83c004190 con 0x7ff8400047c0 2023-11-24T06:16:29.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.770+0000 7ff851d9b640 1 -- 172.21.15.5:0/3075715471 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff83c004520 con 0x7ff8400047c0 2023-11-24T06:16:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.771+0000 7ff8588ee640 1 -- 172.21.15.5:0/3075715471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8400047c0 msgr2=0x7ff840004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.771+0000 7ff8588ee640 1 --2- 172.21.15.5:0/3075715471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8400047c0 0x7ff840004c40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff854144190 tx=0x7ff8400057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.771+0000 7ff852d9d640 1 -- 172.21.15.5:0/3075715471 reap_dead start 2023-11-24T06:16:29.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.771+0000 7ff8588ee640 1 -- 172.21.15.5:0/3075715471 shutdown_connections 2023-11-24T06:16:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.771+0000 7ff8588ee640 1 -- 172.21.15.5:0/3075715471 >> 172.21.15.5:0/3075715471 conn(0x7ff8540d0040 msgr2=0x7ff8540d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.771+0000 7ff8588ee640 1 -- 172.21.15.5:0/3075715471 shutdown_connections 2023-11-24T06:16:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.771+0000 7ff8588ee640 1 -- 172.21.15.5:0/3075715471 wait complete. 2023-11-24T06:16:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.773+0000 7ff8588ee640 1 Processor -- start 2023-11-24T06:16:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.773+0000 7ff8588ee640 1 -- start start 2023-11-24T06:16:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.774+0000 7ff8588ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8400047c0 0x7ff85406ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.774+0000 7ff8588ee640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8540c9e00 0x7ff8540ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.774+0000 7ff8588ee640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff854152600 0x7ff854152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.774+0000 7ff8588ee640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85413d930 con 0x7ff8400047c0 2023-11-24T06:16:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.774+0000 7ff8588ee640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540ca7c0 con 0x7ff854152600 2023-11-24T06:16:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.774+0000 7ff8588ee640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff854159fc0 con 0x7ff8540c9e00 2023-11-24T06:16:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.774+0000 7ff85259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8400047c0 0x7ff85406ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.774+0000 7ff85259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8400047c0 0x7ff85406ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40476/0 (socket says 172.21.15.5:40476) 2023-11-24T06:16:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.774+0000 7ff85259c640 1 -- 172.21.15.5:0/56399542 learned_addr learned my addr 172.21.15.5:0/56399542 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.774+0000 7ff849d9b640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8540c9e00 0x7ff8540ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.774+0000 7ff85259c640 1 -- 172.21.15.5:0/56399542 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8540c9e00 msgr2=0x7ff8540ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.774+0000 7ff852d9d640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff854152600 0x7ff854152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.774+0000 7ff85259c640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8540c9e00 0x7ff8540ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.775+0000 7ff85259c640 1 -- 172.21.15.5:0/56399542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff854152600 msgr2=0x7ff854152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.775+0000 7ff85259c640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff854152600 0x7ff854152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.775+0000 7ff85259c640 1 -- 172.21.15.5:0/56399542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8540ca9e0 con 0x7ff8400047c0 2023-11-24T06:16:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.775+0000 7ff852d9d640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff854152600 0x7ff854152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.775+0000 7ff85259c640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8400047c0 0x7ff85406ebd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff844010a30 tx=0x7ff844010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.775+0000 7ff84b7fe640 1 -- 172.21.15.5:0/56399542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84400ec80 con 0x7ff8400047c0 2023-11-24T06:16:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.775+0000 7ff84b7fe640 1 -- 172.21.15.5:0/56399542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84400ede0 con 0x7ff8400047c0 2023-11-24T06:16:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.775+0000 7ff84b7fe640 1 -- 172.21.15.5:0/56399542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84401c620 con 0x7ff8400047c0 2023-11-24T06:16:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.775+0000 7ff8588ee640 1 -- 172.21.15.5:0/56399542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8540cacc0 con 0x7ff8400047c0 2023-11-24T06:16:29.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.775+0000 7ff8588ee640 1 -- 172.21.15.5:0/56399542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85413db40 con 0x7ff8400047c0 2023-11-24T06:16:29.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.777+0000 7ff8588ee640 1 -- 172.21.15.5:0/56399542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8540cae50 con 0x7ff8400047c0 2023-11-24T06:16:29.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.777+0000 7ff84b7fe640 1 -- 172.21.15.5:0/56399542 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff84401c780 con 0x7ff8400047c0 2023-11-24T06:16:29.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.779+0000 7ff84b7fe640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff83406ae70 0x7ff83406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:29.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.779+0000 7ff84b7fe640 1 -- 172.21.15.5:0/56399542 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff844089cb0 con 0x7ff8400047c0 2023-11-24T06:16:29.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.779+0000 7ff849d9b640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff83406ae70 0x7ff83406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:29.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.780+0000 7ff849d9b640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff83406ae70 0x7ff83406d330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff83c035000 tx=0x7ff83c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:29.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.783+0000 7ff84b7fe640 1 -- 172.21.15.5:0/56399542 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff844054cc0 con 0x7ff8400047c0 2023-11-24T06:16:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.983+0000 7ff8588ee640 1 -- 172.21.15.5:0/56399542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff85406e750 con 0x7ff8400047c0 2023-11-24T06:16:29.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.984+0000 7ff84b7fe640 1 -- 172.21.15.5:0/56399542 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff8440582e0 con 0x7ff8400047c0 2023-11-24T06:16:29.987 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:29.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.987+0000 7ff8588ee640 1 -- 172.21.15.5:0/56399542 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff83406ae70 msgr2=0x7ff83406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:29.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.987+0000 7ff8588ee640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff83406ae70 0x7ff83406d330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff83c035000 tx=0x7ff83c030000 comp rx=0 tx=0).stop 2023-11-24T06:16:29.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.987+0000 7ff8588ee640 1 -- 172.21.15.5:0/56399542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8400047c0 msgr2=0x7ff85406ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:29.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.987+0000 7ff8588ee640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8400047c0 0x7ff85406ebd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff844010a30 tx=0x7ff844010ef0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.987+0000 7ff8588ee640 1 -- 172.21.15.5:0/56399542 shutdown_connections 2023-11-24T06:16:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.987+0000 7ff8588ee640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff83406ae70 0x7ff83406d330 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.987+0000 7ff8588ee640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff8540c9e00 0x7ff8540ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.987+0000 7ff8588ee640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff854152600 0x7ff854152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.987+0000 7ff8588ee640 1 --2- 172.21.15.5:0/56399542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8400047c0 0x7ff85406ebd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.987+0000 7ff8588ee640 1 -- 172.21.15.5:0/56399542 >> 172.21.15.5:0/56399542 conn(0x7ff8540d0040 msgr2=0x7ff85406db50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.987+0000 7ff8588ee640 1 -- 172.21.15.5:0/56399542 shutdown_connections 2023-11-24T06:16:29.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:29.987+0000 7ff8588ee640 1 -- 172.21.15.5:0/56399542 wait complete. 2023-11-24T06:16:30.011 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.241+0000 7f75d2f06640 1 Processor -- start 2023-11-24T06:16:30.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.242+0000 7f75d2f06640 1 -- start start 2023-11-24T06:16:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.243+0000 7f75d2f06640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f75cc0d1d90 0x7f75cc0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.243+0000 7f75d2f06640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75cc0c6bf0 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.243+0000 7f75d2f06640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75cc0d13f0 con 0x7f75cc15d3b0 2023-11-24T06:16:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.243+0000 7f75d2f06640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75cc0cedc0 con 0x7f75cc0d1d90 2023-11-24T06:16:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.244+0000 7f75d2705640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f75cc0d0fa0 0x7f75cc0b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35524/0 (socket says 172.21.15.5:35524) 2023-11-24T06:16:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.244+0000 7f75d2705640 1 -- 172.21.15.5:0/3397959115 learned_addr learned my addr 172.21.15.5:0/3397959115 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.244+0000 7f75d1703640 1 --2- 172.21.15.5:0/3397959115 >> v2:172.21.15.6:3300/0 conn(0x7f75cc0d1d90 0x7f75cc0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:30.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.244+0000 7f75d0f02640 1 -- 172.21.15.5:0/3397959115 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971194673 0 0) 0x7f75cc0c6bf0 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.245+0000 7f75d0f02640 1 -- 172.21.15.5:0/3397959115 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75b8003520 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.245+0000 7f75d1703640 1 -- 172.21.15.5:0/3397959115 >> v1:172.21.15.6:6789/0 conn(0x7f75cc15d3b0 legacy=0x7f75cc0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:30.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.245+0000 7f75d0f02640 1 -- 172.21.15.5:0/3397959115 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1767614582 0 0) 0x7f75b8003520 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.245+0000 7f75d1703640 1 -- 172.21.15.5:0/3397959115 >> v1:172.21.15.5:6789/0 conn(0x7f75cc0d0fa0 legacy=0x7f75cc0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:30.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.245+0000 7f75d1703640 1 -- 172.21.15.5:0/3397959115 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc0c6e40 con 0x7f75cc0d1d90 2023-11-24T06:16:30.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.246+0000 7f75d1703640 1 --2- 172.21.15.5:0/3397959115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75cc0d1d90 0x7f75cc0ce6c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0097a0 tx=0x7f75bc02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfcb17645db00279 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.247+0000 7f75d0f02640 1 -- 172.21.15.5:0/3397959115 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4001670 con 0x7f75cc0d1d90 2023-11-24T06:16:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.247+0000 7f75d0f02640 1 -- 172.21.15.5:0/3397959115 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75bc02e050 con 0x7f75cc0d1d90 2023-11-24T06:16:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.247+0000 7f75d0f02640 1 -- 172.21.15.5:0/3397959115 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc030730 con 0x7f75cc0d1d90 2023-11-24T06:16:30.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.247+0000 7f75d2f06640 1 -- 172.21.15.5:0/3397959115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75cc0d1d90 msgr2=0x7f75cc0ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:30.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.247+0000 7f75d2f06640 1 --2- 172.21.15.5:0/3397959115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75cc0d1d90 0x7f75cc0ce6c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0097a0 tx=0x7f75bc02b790 comp rx=0 tx=0).stop 2023-11-24T06:16:30.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.247+0000 7f75d2f06640 1 -- 172.21.15.5:0/3397959115 shutdown_connections 2023-11-24T06:16:30.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.247+0000 7f75d2f06640 1 --2- 172.21.15.5:0/3397959115 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75cc0d1d90 0x7f75cc0ce6c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.247+0000 7f75d2f06640 1 -- 172.21.15.5:0/3397959115 >> 172.21.15.5:0/3397959115 conn(0x7f75cc0d30a0 msgr2=0x7f75cc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:30.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.248+0000 7f75d2f06640 1 -- 172.21.15.5:0/3397959115 shutdown_connections 2023-11-24T06:16:30.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.248+0000 7f75d2f06640 1 -- 172.21.15.5:0/3397959115 wait complete. 2023-11-24T06:16:30.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.249+0000 7f75d2f06640 1 Processor -- start 2023-11-24T06:16:30.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.249+0000 7f75d2f06640 1 -- start start 2023-11-24T06:16:30.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.250+0000 7f75d2f06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc0d0fa0 0x7f75cc157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:30.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.250+0000 7f75d2f06640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75cc0d1d90 0x7f75cc157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:30.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.250+0000 7f75d2f06640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75cc15d3b0 0x7f75cc158370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:30.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.250+0000 7f75d2f06640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0cb840 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.250+0000 7f75d2f06640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0caca0 con 0x7f75cc0d1d90 2023-11-24T06:16:30.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.250+0000 7f75d2f06640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc0caf10 con 0x7f75cc15d3b0 2023-11-24T06:16:30.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.251+0000 7f75d1f04640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc0d0fa0 0x7f75cc157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:30.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.251+0000 7f75d1f04640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc0d0fa0 0x7f75cc157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40478/0 (socket says 172.21.15.5:40478) 2023-11-24T06:16:30.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.251+0000 7f75d1f04640 1 -- 172.21.15.5:0/2018092838 learned_addr learned my addr 172.21.15.5:0/2018092838 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:30.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.251+0000 7f75d1703640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75cc0d1d90 0x7f75cc157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:30.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.251+0000 7f75d1f04640 1 -- 172.21.15.5:0/2018092838 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75cc15d3b0 msgr2=0x7f75cc158370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.251+0000 7f75d2705640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75cc15d3b0 0x7f75cc158370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.251+0000 7f75d1f04640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75cc15d3b0 0x7f75cc158370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.251+0000 7f75d1f04640 1 -- 172.21.15.5:0/2018092838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75cc0d1d90 msgr2=0x7f75cc157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.251+0000 7f75d1f04640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75cc0d1d90 0x7f75cc157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.251+0000 7f75d1f04640 1 -- 172.21.15.5:0/2018092838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc009400 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.251+0000 7f75d1703640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75cc0d1d90 0x7f75cc157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.251+0000 7f75d2705640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75cc15d3b0 0x7f75cc158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.252+0000 7f75d1f04640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc0d0fa0 0x7f75cc157610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f75c800c760 tx=0x7f75c800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.252+0000 7f75c27fc640 1 -- 172.21.15.5:0/2018092838 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8003b70 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.252+0000 7f75c27fc640 1 -- 172.21.15.5:0/2018092838 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c8003cf0 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.252+0000 7f75c27fc640 1 -- 172.21.15.5:0/2018092838 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c800aa90 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.252+0000 7f75d2f06640 1 -- 172.21.15.5:0/2018092838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc0cb1f0 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.252+0000 7f75d2f06640 1 -- 172.21.15.5:0/2018092838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc157b50 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.253+0000 7f75d2f06640 1 -- 172.21.15.5:0/2018092838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc04e840 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.254+0000 7f75c27fc640 1 -- 172.21.15.5:0/2018092838 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f75c800abf0 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.256+0000 7f75c27fc640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f75b006ae70 0x7f75b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.256+0000 7f75c27fc640 1 -- 172.21.15.5:0/2018092838 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75c80894b0 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.256+0000 7f75d1703640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f75b006ae70 0x7f75b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:30.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.257+0000 7f75d1703640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f75b006ae70 0x7f75b006d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75bc034000 tx=0x7f75bc02f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.261+0000 7f75c27fc640 1 -- 172.21.15.5:0/2018092838 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f75c80544c0 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.458+0000 7f75d2f06640 1 -- 172.21.15.5:0/2018092838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f75cc0c8fa0 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.459+0000 7f75c27fc640 1 -- 172.21.15.5:0/2018092838 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f75c8003e60 con 0x7f75cc0d0fa0 2023-11-24T06:16:30.462 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.462+0000 7f75d2f06640 1 -- 172.21.15.5:0/2018092838 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f75b006ae70 msgr2=0x7f75b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.462+0000 7f75d2f06640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f75b006ae70 0x7f75b006d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75bc034000 tx=0x7f75bc02f000 comp rx=0 tx=0).stop 2023-11-24T06:16:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.462+0000 7f75d2f06640 1 -- 172.21.15.5:0/2018092838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc0d0fa0 msgr2=0x7f75cc157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.462+0000 7f75d2f06640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc0d0fa0 0x7f75cc157610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f75c800c760 tx=0x7f75c800cc20 comp rx=0 tx=0).stop 2023-11-24T06:16:30.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.462+0000 7f75d2f06640 1 -- 172.21.15.5:0/2018092838 shutdown_connections 2023-11-24T06:16:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.462+0000 7f75d2f06640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f75b006ae70 0x7f75b006d330 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.462+0000 7f75d2f06640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f75cc15d3b0 0x7f75cc158370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.462+0000 7f75d2f06640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75cc0d1d90 0x7f75cc157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.462+0000 7f75d2f06640 1 --2- 172.21.15.5:0/2018092838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc0d0fa0 0x7f75cc157610 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.462+0000 7f75d2f06640 1 -- 172.21.15.5:0/2018092838 >> 172.21.15.5:0/2018092838 conn(0x7f75cc0d30a0 msgr2=0x7f75cc06c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.462+0000 7f75d2f06640 1 -- 172.21.15.5:0/2018092838 shutdown_connections 2023-11-24T06:16:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.462+0000 7f75d2f06640 1 -- 172.21.15.5:0/2018092838 wait complete. 2023-11-24T06:16:30.483 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:30.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.714+0000 7f751944b640 1 Processor -- start 2023-11-24T06:16:30.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.714+0000 7f751944b640 1 -- start start 2023-11-24T06:16:30.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.715+0000 7f751944b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f751415d3b0 0x7f75140ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:30.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.716+0000 7f751944b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75140c6bf0 con 0x7f75140d0fa0 2023-11-24T06:16:30.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.716+0000 7f751944b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75140d1430 con 0x7f75140d1d90 2023-11-24T06:16:30.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.716+0000 7f751944b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75140cedc0 con 0x7f751415d3b0 2023-11-24T06:16:30.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.716+0000 7f75137fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f75140d0fa0 0x7f75140d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35540/0 (socket says 172.21.15.5:35540) 2023-11-24T06:16:30.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.716+0000 7f75137fe640 1 -- 172.21.15.5:0/3257894823 learned_addr learned my addr 172.21.15.5:0/3257894823 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.716+0000 7f7513fff640 1 --2- 172.21.15.5:0/3257894823 >> v2:172.21.15.6:3300/0 conn(0x7f751415d3b0 0x7f75140ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.716+0000 7f7512ffd640 1 -- 172.21.15.5:0/3257894823 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420905029 0 0) 0x7f75140c6bf0 con 0x7f75140d0fa0 2023-11-24T06:16:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.717+0000 7f7512ffd640 1 -- 172.21.15.5:0/3257894823 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74fc003520 con 0x7f75140d0fa0 2023-11-24T06:16:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.717+0000 7f7513fff640 1 -- 172.21.15.5:0/3257894823 >> v1:172.21.15.6:6789/0 conn(0x7f75140d1d90 legacy=0x7f75140b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.717+0000 7f7513fff640 1 -- 172.21.15.5:0/3257894823 >> v1:172.21.15.5:6789/0 conn(0x7f75140d0fa0 legacy=0x7f75140d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.717+0000 7f7513fff640 1 -- 172.21.15.5:0/3257894823 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75140c6e40 con 0x7f751415d3b0 2023-11-24T06:16:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.718+0000 7f7513fff640 1 --2- 172.21.15.5:0/3257894823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f751415d3b0 0x7f75140ce6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7504009d90 tx=0x7f750402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e83f4d8106f92aa7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.718+0000 7f7512ffd640 1 -- 172.21.15.5:0/3257894823 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750402cc60 con 0x7f751415d3b0 2023-11-24T06:16:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.718+0000 7f7512ffd640 1 -- 172.21.15.5:0/3257894823 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750402cdc0 con 0x7f751415d3b0 2023-11-24T06:16:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.718+0000 7f7512ffd640 1 -- 172.21.15.5:0/3257894823 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504034640 con 0x7f751415d3b0 2023-11-24T06:16:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.719+0000 7f751944b640 1 -- 172.21.15.5:0/3257894823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f751415d3b0 msgr2=0x7f75140ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.719+0000 7f751944b640 1 --2- 172.21.15.5:0/3257894823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f751415d3b0 0x7f75140ce6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7504009d90 tx=0x7f750402beb0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.719+0000 7f751944b640 1 -- 172.21.15.5:0/3257894823 shutdown_connections 2023-11-24T06:16:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.719+0000 7f751944b640 1 --2- 172.21.15.5:0/3257894823 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f751415d3b0 0x7f75140ce6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.719+0000 7f751944b640 1 -- 172.21.15.5:0/3257894823 >> 172.21.15.5:0/3257894823 conn(0x7f75140d30a0 msgr2=0x7f75140d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.719+0000 7f751944b640 1 -- 172.21.15.5:0/3257894823 shutdown_connections 2023-11-24T06:16:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.719+0000 7f751944b640 1 -- 172.21.15.5:0/3257894823 wait complete. 2023-11-24T06:16:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.721+0000 7f751944b640 1 Processor -- start 2023-11-24T06:16:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.721+0000 7f751944b640 1 -- start start 2023-11-24T06:16:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.722+0000 7f751944b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75140d0fa0 0x7f75140b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.722+0000 7f751944b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75140d1d90 0x7f75141500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.722+0000 7f751944b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f751415d3b0 0x7f751406c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.722+0000 7f751944b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751414fc50 con 0x7f75140d1d90 2023-11-24T06:16:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.722+0000 7f751944b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751414f040 con 0x7f75140d0fa0 2023-11-24T06:16:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.722+0000 7f751944b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f751414f320 con 0x7f751415d3b0 2023-11-24T06:16:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.722+0000 7f75137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75140d1d90 0x7f75141500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.723+0000 7f75137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75140d1d90 0x7f75141500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40490/0 (socket says 172.21.15.5:40490) 2023-11-24T06:16:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.723+0000 7f75137fe640 1 -- 172.21.15.5:0/2359627556 learned_addr learned my addr 172.21.15.5:0/2359627556 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.723+0000 7f7513fff640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75140d0fa0 0x7f75140b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.723+0000 7f7518c4a640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f751415d3b0 0x7f751406c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.723+0000 7f75137fe640 1 -- 172.21.15.5:0/2359627556 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f751415d3b0 msgr2=0x7f751406c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.723+0000 7f75137fe640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f751415d3b0 0x7f751406c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.723+0000 7f75137fe640 1 -- 172.21.15.5:0/2359627556 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75140d0fa0 msgr2=0x7f75140b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.723+0000 7f75137fe640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75140d0fa0 0x7f75140b82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.723+0000 7f75137fe640 1 -- 172.21.15.5:0/2359627556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7504009a70 con 0x7f75140d1d90 2023-11-24T06:16:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.723+0000 7f7518c4a640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f751415d3b0 0x7f751406c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.723+0000 7f75137fe640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75140d1d90 0x7f75141500e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f750800c500 tx=0x7f750800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.724+0000 7f7510ff9640 1 -- 172.21.15.5:0/2359627556 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508005ce0 con 0x7f75140d1d90 2023-11-24T06:16:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.724+0000 7f7510ff9640 1 -- 172.21.15.5:0/2359627556 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7508005e40 con 0x7f75140d1d90 2023-11-24T06:16:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.724+0000 7f7510ff9640 1 -- 172.21.15.5:0/2359627556 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508007330 con 0x7f75140d1d90 2023-11-24T06:16:30.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.724+0000 7f751944b640 1 -- 172.21.15.5:0/2359627556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f751414f600 con 0x7f75140d1d90 2023-11-24T06:16:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.724+0000 7f751944b640 1 -- 172.21.15.5:0/2359627556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7514150620 con 0x7f75140d1d90 2023-11-24T06:16:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.725+0000 7f751944b640 1 -- 172.21.15.5:0/2359627556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751414fe00 con 0x7f75140d1d90 2023-11-24T06:16:30.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.726+0000 7f7510ff9640 1 -- 172.21.15.5:0/2359627556 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7508007550 con 0x7f75140d1d90 2023-11-24T06:16:30.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.727+0000 7f7510ff9640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f750006ae40 0x7f750006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:30.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.727+0000 7f7510ff9640 1 -- 172.21.15.5:0/2359627556 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7508088c80 con 0x7f75140d1d90 2023-11-24T06:16:30.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.727+0000 7f7513fff640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f750006ae40 0x7f750006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:30.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.728+0000 7f7513fff640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f750006ae40 0x7f750006d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7504036000 tx=0x7f7504030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:30.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.731+0000 7f7510ff9640 1 -- 172.21.15.5:0/2359627556 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f751414fe00 con 0x7f75140d1d90 2023-11-24T06:16:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.926+0000 7f751944b640 1 -- 172.21.15.5:0/2359627556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f751404e840 con 0x7f75140d1d90 2023-11-24T06:16:30.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.927+0000 7f7510ff9640 1 -- 172.21.15.5:0/2359627556 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7508017020 con 0x7f75140d1d90 2023-11-24T06:16:30.930 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:30.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.930+0000 7f751944b640 1 -- 172.21.15.5:0/2359627556 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f750006ae40 msgr2=0x7f750006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:30.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.930+0000 7f751944b640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f750006ae40 0x7f750006d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7504036000 tx=0x7f7504030000 comp rx=0 tx=0).stop 2023-11-24T06:16:30.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.930+0000 7f751944b640 1 -- 172.21.15.5:0/2359627556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75140d1d90 msgr2=0x7f75141500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:30.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.930+0000 7f751944b640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75140d1d90 0x7f75141500e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f750800c500 tx=0x7f750800c9c0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.930+0000 7f751944b640 1 -- 172.21.15.5:0/2359627556 shutdown_connections 2023-11-24T06:16:30.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.930+0000 7f751944b640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f750006ae40 0x7f750006d300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.930+0000 7f751944b640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f751415d3b0 0x7f751406c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.930+0000 7f751944b640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75140d1d90 0x7f75141500e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.930+0000 7f751944b640 1 --2- 172.21.15.5:0/2359627556 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f75140d0fa0 0x7f75140b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:30.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.930+0000 7f751944b640 1 -- 172.21.15.5:0/2359627556 >> 172.21.15.5:0/2359627556 conn(0x7f75140d30a0 msgr2=0x7f75140c6410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:30.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.931+0000 7f751944b640 1 -- 172.21.15.5:0/2359627556 shutdown_connections 2023-11-24T06:16:30.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:30.931+0000 7f751944b640 1 -- 172.21.15.5:0/2359627556 wait complete. 2023-11-24T06:16:30.955 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:31.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.168+0000 7f7c5cdba640 1 Processor -- start 2023-11-24T06:16:31.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.168+0000 7f7c5cdba640 1 -- start start 2023-11-24T06:16:31.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.170+0000 7f7c5cdba640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7c5813dfd0 0x7f7c5815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:31.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.170+0000 7f7c5cdba640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c580cf050 0x7f7c580cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:31.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.170+0000 7f7c5cdba640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7c5806c130 0x7f7c5806c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:31.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.170+0000 7f7c5cdba640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c580cfa10 con 0x7f7c5813dfd0 2023-11-24T06:16:31.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.170+0000 7f7c5cdba640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c5815fa70 con 0x7f7c5806c130 2023-11-24T06:16:31.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.170+0000 7f7c5cdba640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5815ee60 con 0x7f7c580cf050 2023-11-24T06:16:31.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.171+0000 7f7c577fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7c5813dfd0 0x7f7c5815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:31.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.171+0000 7f7c577fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7c5813dfd0 0x7f7c5815be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40506/0 (socket says 172.21.15.5:40506) 2023-11-24T06:16:31.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.171+0000 7f7c577fe640 1 -- 172.21.15.5:0/1591703698 learned_addr learned my addr 172.21.15.5:0/1591703698 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.171+0000 7f7c577fe640 1 -- 172.21.15.5:0/1591703698 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c580cf050 msgr2=0x7f7c580cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.171+0000 7f7c577fe640 1 --2- 172.21.15.5:0/1591703698 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c580cf050 0x7f7c580cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.171+0000 7f7c57fff640 1 --2- 172.21.15.5:0/1591703698 >> v2:172.21.15.6:3300/0 conn(0x7f7c5806c130 0x7f7c5806c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.171+0000 7f7c577fe640 1 -- 172.21.15.5:0/1591703698 >> v2:172.21.15.6:3300/0 conn(0x7f7c5806c130 msgr2=0x7f7c5806c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.171+0000 7f7c577fe640 1 --2- 172.21.15.5:0/1591703698 >> v2:172.21.15.6:3300/0 conn(0x7f7c5806c130 0x7f7c5806c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.171+0000 7f7c577fe640 1 -- 172.21.15.5:0/1591703698 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5815f140 con 0x7f7c5813dfd0 2023-11-24T06:16:31.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.173+0000 7f7c577fe640 1 --2- 172.21.15.5:0/1591703698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c5813dfd0 0x7f7c5815be70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7c5806d520 tx=0x7f7c4802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e94a0fe4d58e46fb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:31.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.173+0000 7f7c567fc640 1 -- 172.21.15.5:0/1591703698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48031870 con 0x7f7c5813dfd0 2023-11-24T06:16:31.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.173+0000 7f7c567fc640 1 -- 172.21.15.5:0/1591703698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c480319d0 con 0x7f7c5813dfd0 2023-11-24T06:16:31.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.173+0000 7f7c567fc640 1 -- 172.21.15.5:0/1591703698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48031d60 con 0x7f7c5813dfd0 2023-11-24T06:16:31.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.174+0000 7f7c5cdba640 1 -- 172.21.15.5:0/1591703698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c5813dfd0 msgr2=0x7f7c5815be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:31.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.174+0000 7f7c5cdba640 1 --2- 172.21.15.5:0/1591703698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c5813dfd0 0x7f7c5815be70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7c5806d520 tx=0x7f7c4802b910 comp rx=0 tx=0).stop 2023-11-24T06:16:31.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.174+0000 7f7c5cdba640 1 -- 172.21.15.5:0/1591703698 shutdown_connections 2023-11-24T06:16:31.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.174+0000 7f7c5cdba640 1 --2- 172.21.15.5:0/1591703698 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c580cf050 0x7f7c580cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.174+0000 7f7c5cdba640 1 --2- 172.21.15.5:0/1591703698 >> v2:172.21.15.6:3300/0 conn(0x7f7c5806c130 0x7f7c5806c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.174+0000 7f7c5cdba640 1 --2- 172.21.15.5:0/1591703698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c5813dfd0 0x7f7c5815be70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.174+0000 7f7c5cdba640 1 -- 172.21.15.5:0/1591703698 >> 172.21.15.5:0/1591703698 conn(0x7f7c5806a860 msgr2=0x7f7c5806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:31.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.174+0000 7f7c5cdba640 1 -- 172.21.15.5:0/1591703698 shutdown_connections 2023-11-24T06:16:31.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.174+0000 7f7c5cdba640 1 -- 172.21.15.5:0/1591703698 wait complete. 2023-11-24T06:16:31.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.176+0000 7f7c5cdba640 1 Processor -- start 2023-11-24T06:16:31.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.176+0000 7f7c5cdba640 1 -- start start 2023-11-24T06:16:31.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.177+0000 7f7c5cdba640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5806c130 0x7f7c58155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:31.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.177+0000 7f7c5cdba640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c580cf050 0x7f7c58157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:31.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.177+0000 7f7c5cdba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c5813dfd0 0x7f7c58156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:31.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.177+0000 7f7c5cdba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c58155c00 con 0x7f7c5813dfd0 2023-11-24T06:16:31.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.177+0000 7f7c5cdba640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c58157f50 con 0x7f7c5806c130 2023-11-24T06:16:31.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.177+0000 7f7c5cdba640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c58158230 con 0x7f7c580cf050 2023-11-24T06:16:31.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.177+0000 7f7c57fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c5813dfd0 0x7f7c58156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:31.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.177+0000 7f7c57fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c5813dfd0 0x7f7c58156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40522/0 (socket says 172.21.15.5:40522) 2023-11-24T06:16:31.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.177+0000 7f7c57fff640 1 -- 172.21.15.5:0/118337574 learned_addr learned my addr 172.21.15.5:0/118337574 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:31.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.178+0000 7f7c577fe640 1 --2- 172.21.15.5:0/118337574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5806c130 0x7f7c58155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:31.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.178+0000 7f7c56ffd640 1 --2- 172.21.15.5:0/118337574 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c580cf050 0x7f7c58157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:31.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.178+0000 7f7c57fff640 1 -- 172.21.15.5:0/118337574 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c580cf050 msgr2=0x7f7c58157620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:31.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.178+0000 7f7c57fff640 1 --2- 172.21.15.5:0/118337574 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c580cf050 0x7f7c58157620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.178+0000 7f7c57fff640 1 -- 172.21.15.5:0/118337574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5806c130 msgr2=0x7f7c58155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:31.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.178+0000 7f7c57fff640 1 --2- 172.21.15.5:0/118337574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5806c130 0x7f7c58155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.178+0000 7f7c57fff640 1 -- 172.21.15.5:0/118337574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c480094d0 con 0x7f7c5813dfd0 2023-11-24T06:16:31.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.178+0000 7f7c57fff640 1 --2- 172.21.15.5:0/118337574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c5813dfd0 0x7f7c58156650 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000cd70 tx=0x7f7c5000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:31.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.179+0000 7f7c3ffff640 1 -- 172.21.15.5:0/118337574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50003b90 con 0x7f7c5813dfd0 2023-11-24T06:16:31.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.179+0000 7f7c3ffff640 1 -- 172.21.15.5:0/118337574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c50003cf0 con 0x7f7c5813dfd0 2023-11-24T06:16:31.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.179+0000 7f7c3ffff640 1 -- 172.21.15.5:0/118337574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50004240 con 0x7f7c5813dfd0 2023-11-24T06:16:31.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.179+0000 7f7c5cdba640 1 -- 172.21.15.5:0/118337574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c58155e50 con 0x7f7c5813dfd0 2023-11-24T06:16:31.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.179+0000 7f7c5cdba640 1 -- 172.21.15.5:0/118337574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c581556b0 con 0x7f7c5813dfd0 2023-11-24T06:16:31.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.180+0000 7f7c5cdba640 1 -- 172.21.15.5:0/118337574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c58156370 con 0x7f7c5813dfd0 2023-11-24T06:16:31.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.181+0000 7f7c3ffff640 1 -- 172.21.15.5:0/118337574 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c500043a0 con 0x7f7c5813dfd0 2023-11-24T06:16:31.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.182+0000 7f7c3ffff640 1 --2- 172.21.15.5:0/118337574 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7c4406ae40 0x7f7c4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:31.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.182+0000 7f7c3ffff640 1 -- 172.21.15.5:0/118337574 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c50012070 con 0x7f7c5813dfd0 2023-11-24T06:16:31.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.182+0000 7f7c577fe640 1 --2- 172.21.15.5:0/118337574 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7c4406ae40 0x7f7c4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:31.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.183+0000 7f7c577fe640 1 --2- 172.21.15.5:0/118337574 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7c4406ae40 0x7f7c4406d300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c48036000 tx=0x7f7c4803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:31.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.186+0000 7f7c3ffff640 1 -- 172.21.15.5:0/118337574 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c58156370 con 0x7f7c5813dfd0 2023-11-24T06:16:31.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.382+0000 7f7c5cdba640 1 -- 172.21.15.5:0/118337574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c58156370 con 0x7f7c5813dfd0 2023-11-24T06:16:31.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.385+0000 7f7c3ffff640 1 -- 172.21.15.5:0/118337574 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16361 (secure 0 0 0) 0x7f7c58156370 con 0x7f7c5813dfd0 2023-11-24T06:16:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:31.388 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":22,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:15:52.779059+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.388+0000 7f7c5cdba640 1 -- 172.21.15.5:0/118337574 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7c4406ae40 msgr2=0x7f7c4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.388+0000 7f7c5cdba640 1 --2- 172.21.15.5:0/118337574 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7c4406ae40 0x7f7c4406d300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c48036000 tx=0x7f7c4803c040 comp rx=0 tx=0).stop 2023-11-24T06:16:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.388+0000 7f7c5cdba640 1 -- 172.21.15.5:0/118337574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c5813dfd0 msgr2=0x7f7c58156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.388+0000 7f7c5cdba640 1 --2- 172.21.15.5:0/118337574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c5813dfd0 0x7f7c58156650 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000cd70 tx=0x7f7c5000a660 comp rx=0 tx=0).stop 2023-11-24T06:16:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.388+0000 7f7c5cdba640 1 -- 172.21.15.5:0/118337574 shutdown_connections 2023-11-24T06:16:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.388+0000 7f7c5cdba640 1 --2- 172.21.15.5:0/118337574 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7c4406ae40 0x7f7c4406d300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.388+0000 7f7c5cdba640 1 --2- 172.21.15.5:0/118337574 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c580cf050 0x7f7c58157620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.388+0000 7f7c5cdba640 1 --2- 172.21.15.5:0/118337574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c5813dfd0 0x7f7c58156650 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.388+0000 7f7c5cdba640 1 --2- 172.21.15.5:0/118337574 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c5806c130 0x7f7c58155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.388+0000 7f7c5cdba640 1 -- 172.21.15.5:0/118337574 >> 172.21.15.5:0/118337574 conn(0x7f7c5806a860 msgr2=0x7f7c58071a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.388+0000 7f7c5cdba640 1 -- 172.21.15.5:0/118337574 shutdown_connections 2023-11-24T06:16:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.389+0000 7f7c5cdba640 1 -- 172.21.15.5:0/118337574 wait complete. 2023-11-24T06:16:31.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:31.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:31.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.643+0000 7f1fd08d9640 1 Processor -- start 2023-11-24T06:16:31.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.644+0000 7f1fd08d9640 1 -- start start 2023-11-24T06:16:31.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.645+0000 7f1fd08d9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1fcc0b4140 0x7f1fcc071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:31.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.645+0000 7f1fd08d9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc070d40 con 0x7f1fcc1439b0 2023-11-24T06:16:31.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.645+0000 7f1fd08d9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc071020 con 0x7f1fcc1435d0 2023-11-24T06:16:31.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.645+0000 7f1fd08d9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc071300 con 0x7f1fcc0b4140 2023-11-24T06:16:31.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.646+0000 7f1fcb59e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f1fcc1439b0 0x7f1fcc070050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35544/0 (socket says 172.21.15.5:35544) 2023-11-24T06:16:31.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.646+0000 7f1fcb59e640 1 -- 172.21.15.5:0/3972511014 learned_addr learned my addr 172.21.15.5:0/3972511014 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:31.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.646+0000 7f1fc3fff640 1 --2- 172.21.15.5:0/3972511014 >> v2:172.21.15.6:3300/0 conn(0x7f1fcc0b4140 0x7f1fcc071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:31.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.647+0000 7f1fca59c640 1 -- 172.21.15.5:0/3972511014 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366654346 0 0) 0x7f1fcc070d40 con 0x7f1fcc1439b0 2023-11-24T06:16:31.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.647+0000 7f1fca59c640 1 -- 172.21.15.5:0/3972511014 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fb8003520 con 0x7f1fcc1439b0 2023-11-24T06:16:31.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.647+0000 7f1fc3fff640 1 -- 172.21.15.5:0/3972511014 >> v1:172.21.15.6:6789/0 conn(0x7f1fcc1435d0 legacy=0x7f1fcc0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:31.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.647+0000 7f1fc3fff640 1 -- 172.21.15.5:0/3972511014 >> v1:172.21.15.5:6789/0 conn(0x7f1fcc1439b0 legacy=0x7f1fcc070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:31.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.647+0000 7f1fc3fff640 1 -- 172.21.15.5:0/3972511014 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc0715e0 con 0x7f1fcc0b4140 2023-11-24T06:16:31.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.648+0000 7f1fc3fff640 1 --2- 172.21.15.5:0/3972511014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fcc0b4140 0x7f1fcc071de0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40097a0 tx=0x7f1fb402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5df5f7a006d4ef1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.649+0000 7f1fca59c640 1 -- 172.21.15.5:0/3972511014 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4031830 con 0x7f1fcc0b4140 2023-11-24T06:16:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.650+0000 7f1fca59c640 1 -- 172.21.15.5:0/3972511014 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb4031990 con 0x7f1fcc0b4140 2023-11-24T06:16:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.650+0000 7f1fca59c640 1 -- 172.21.15.5:0/3972511014 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4031d20 con 0x7f1fcc0b4140 2023-11-24T06:16:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.650+0000 7f1fd08d9640 1 -- 172.21.15.5:0/3972511014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fcc0b4140 msgr2=0x7f1fcc071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.650+0000 7f1fd08d9640 1 --2- 172.21.15.5:0/3972511014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fcc0b4140 0x7f1fcc071de0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40097a0 tx=0x7f1fb402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.650+0000 7f1fd08d9640 1 -- 172.21.15.5:0/3972511014 shutdown_connections 2023-11-24T06:16:31.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.650+0000 7f1fd08d9640 1 --2- 172.21.15.5:0/3972511014 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fcc0b4140 0x7f1fcc071de0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.650+0000 7f1fd08d9640 1 -- 172.21.15.5:0/3972511014 >> 172.21.15.5:0/3972511014 conn(0x7f1fcc0cec10 msgr2=0x7f1fcc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:31.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.650+0000 7f1fd08d9640 1 -- 172.21.15.5:0/3972511014 shutdown_connections 2023-11-24T06:16:31.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.650+0000 7f1fd08d9640 1 -- 172.21.15.5:0/3972511014 wait complete. 2023-11-24T06:16:31.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.652+0000 7f1fd08d9640 1 Processor -- start 2023-11-24T06:16:31.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.652+0000 7f1fd08d9640 1 -- start start 2023-11-24T06:16:31.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.653+0000 7f1fd08d9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1fcc0b4140 0x7f1fcc139870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:31.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.653+0000 7f1fd08d9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fcc1435d0 0x7f1fcc13e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:31.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.653+0000 7f1fd08d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fcc1439b0 0x7f1fcc0cd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.653+0000 7f1fd08d9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc13cdd0 con 0x7f1fcc1439b0 2023-11-24T06:16:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.653+0000 7f1fd08d9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0cc950 con 0x7f1fcc1435d0 2023-11-24T06:16:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.653+0000 7f1fd08d9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0ccc30 con 0x7f1fcc0b4140 2023-11-24T06:16:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.654+0000 7f1fcb59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fcc1439b0 0x7f1fcc0cd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.654+0000 7f1fcb59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fcc1439b0 0x7f1fcc0cd9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40538/0 (socket says 172.21.15.5:40538) 2023-11-24T06:16:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.654+0000 7f1fcb59e640 1 -- 172.21.15.5:0/2840494490 learned_addr learned my addr 172.21.15.5:0/2840494490 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.654+0000 7f1fc3fff640 1 --2- 172.21.15.5:0/2840494490 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fcc1435d0 0x7f1fcc13e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.654+0000 7f1fcad9d640 1 --2- 172.21.15.5:0/2840494490 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1fcc0b4140 0x7f1fcc139870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.654+0000 7f1fcb59e640 1 -- 172.21.15.5:0/2840494490 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1fcc0b4140 msgr2=0x7f1fcc139870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.654+0000 7f1fcb59e640 1 --2- 172.21.15.5:0/2840494490 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1fcc0b4140 0x7f1fcc139870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.654+0000 7f1fcb59e640 1 -- 172.21.15.5:0/2840494490 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fcc1435d0 msgr2=0x7f1fcc13e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.654+0000 7f1fcb59e640 1 --2- 172.21.15.5:0/2840494490 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fcc1435d0 0x7f1fcc13e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.654+0000 7f1fcb59e640 1 -- 172.21.15.5:0/2840494490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb4009400 con 0x7f1fcc1439b0 2023-11-24T06:16:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.655+0000 7f1fcb59e640 1 --2- 172.21.15.5:0/2840494490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fcc1439b0 0x7f1fcc0cd9f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400ca70 tx=0x7f1fc400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.655+0000 7f1fc37fe640 1 -- 172.21.15.5:0/2840494490 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc400abd0 con 0x7f1fcc1439b0 2023-11-24T06:16:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.655+0000 7f1fc37fe640 1 -- 172.21.15.5:0/2840494490 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc400ad30 con 0x7f1fcc1439b0 2023-11-24T06:16:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.655+0000 7f1fd08d9640 1 -- 172.21.15.5:0/2840494490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc0ccf10 con 0x7f1fcc1439b0 2023-11-24T06:16:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.655+0000 7f1fc37fe640 1 -- 172.21.15.5:0/2840494490 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4012600 con 0x7f1fcc1439b0 2023-11-24T06:16:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.655+0000 7f1fd08d9640 1 -- 172.21.15.5:0/2840494490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc0cd4d0 con 0x7f1fcc1439b0 2023-11-24T06:16:31.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.656+0000 7f1fd08d9640 1 -- 172.21.15.5:0/2840494490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc0cd6c0 con 0x7f1fcc1439b0 2023-11-24T06:16:31.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.657+0000 7f1fc37fe640 1 -- 172.21.15.5:0/2840494490 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1fc4007070 con 0x7f1fcc1439b0 2023-11-24T06:16:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.659+0000 7f1fc37fe640 1 --2- 172.21.15.5:0/2840494490 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1fac06ae40 0x7f1fac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.659+0000 7f1fc37fe640 1 -- 172.21.15.5:0/2840494490 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fc4089510 con 0x7f1fcc1439b0 2023-11-24T06:16:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.659+0000 7f1fcad9d640 1 --2- 172.21.15.5:0/2840494490 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1fac06ae40 0x7f1fac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.660+0000 7f1fcad9d640 1 --2- 172.21.15.5:0/2840494490 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1fac06ae40 0x7f1fac06d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc001640 tx=0x7f1fbc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.663+0000 7f1fc37fe640 1 -- 172.21.15.5:0/2840494490 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1fc40545d0 con 0x7f1fcc1439b0 2023-11-24T06:16:31.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.862+0000 7f1fd08d9640 1 -- 172.21.15.5:0/2840494490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1fcc04e840 con 0x7f1fcc1439b0 2023-11-24T06:16:31.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.863+0000 7f1fc37fe640 1 -- 172.21.15.5:0/2840494490 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1fc4057bf0 con 0x7f1fcc1439b0 2023-11-24T06:16:31.866 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.865+0000 7f1fd08d9640 1 -- 172.21.15.5:0/2840494490 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1fac06ae40 msgr2=0x7f1fac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.865+0000 7f1fd08d9640 1 --2- 172.21.15.5:0/2840494490 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1fac06ae40 0x7f1fac06d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc001640 tx=0x7f1fbc009040 comp rx=0 tx=0).stop 2023-11-24T06:16:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.865+0000 7f1fd08d9640 1 -- 172.21.15.5:0/2840494490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fcc1439b0 msgr2=0x7f1fcc0cd9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.866+0000 7f1fd08d9640 1 --2- 172.21.15.5:0/2840494490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fcc1439b0 0x7f1fcc0cd9f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400ca70 tx=0x7f1fc400cf30 comp rx=0 tx=0).stop 2023-11-24T06:16:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.866+0000 7f1fd08d9640 1 -- 172.21.15.5:0/2840494490 shutdown_connections 2023-11-24T06:16:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.866+0000 7f1fd08d9640 1 --2- 172.21.15.5:0/2840494490 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1fac06ae40 0x7f1fac06d300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.866+0000 7f1fd08d9640 1 --2- 172.21.15.5:0/2840494490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fcc1439b0 0x7f1fcc0cd9f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.866+0000 7f1fd08d9640 1 --2- 172.21.15.5:0/2840494490 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1fcc1435d0 0x7f1fcc13e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.866+0000 7f1fd08d9640 1 --2- 172.21.15.5:0/2840494490 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1fcc0b4140 0x7f1fcc139870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:31.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.866+0000 7f1fd08d9640 1 -- 172.21.15.5:0/2840494490 >> 172.21.15.5:0/2840494490 conn(0x7f1fcc0cec10 msgr2=0x7f1fcc06d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:31.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.866+0000 7f1fd08d9640 1 -- 172.21.15.5:0/2840494490 shutdown_connections 2023-11-24T06:16:31.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:31.866+0000 7f1fd08d9640 1 -- 172.21.15.5:0/2840494490 wait complete. 2023-11-24T06:16:31.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.115+0000 7f33ee89b640 1 Processor -- start 2023-11-24T06:16:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.115+0000 7f33ee89b640 1 -- start start 2023-11-24T06:16:32.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.117+0000 7f33ee89b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f33e815aae0 0x7f33e806cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.117+0000 7f33ee89b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e8157b30 con 0x7f33e80b2f40 2023-11-24T06:16:32.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.117+0000 7f33ee89b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e80b34f0 con 0x7f33e806d2a0 2023-11-24T06:16:32.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.117+0000 7f33ee89b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e815aec0 con 0x7f33e815aae0 2023-11-24T06:16:32.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.117+0000 7f33ece38640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f33e80b2f40 0x7f33e8158310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35556/0 (socket says 172.21.15.5:35556) 2023-11-24T06:16:32.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.117+0000 7f33ece38640 1 -- 172.21.15.5:0/1240619316 learned_addr learned my addr 172.21.15.5:0/1240619316 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.118+0000 7f33e7fff640 1 --2- 172.21.15.5:0/1240619316 >> v2:172.21.15.6:3300/0 conn(0x7f33e815aae0 0x7f33e806cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.118+0000 7f33e77fe640 1 -- 172.21.15.5:0/1240619316 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 964670199 0 0) 0x7f33e8157b30 con 0x7f33e80b2f40 2023-11-24T06:16:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.118+0000 7f33e77fe640 1 -- 172.21.15.5:0/1240619316 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33d4003520 con 0x7f33e80b2f40 2023-11-24T06:16:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.118+0000 7f33e77fe640 1 -- 172.21.15.5:0/1240619316 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1504664535 0 0) 0x7f33d4003520 con 0x7f33e80b2f40 2023-11-24T06:16:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.118+0000 7f33e77fe640 1 -- 172.21.15.5:0/1240619316 >> v2:172.21.15.6:3300/0 conn(0x7f33e815aae0 msgr2=0x7f33e806cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.118+0000 7f33e77fe640 1 --2- 172.21.15.5:0/1240619316 >> v2:172.21.15.6:3300/0 conn(0x7f33e815aae0 0x7f33e806cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.118+0000 7f33e77fe640 1 -- 172.21.15.5:0/1240619316 >> v1:172.21.15.6:6789/0 conn(0x7f33e806d2a0 legacy=0x7f33e80b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.118+0000 7f33e77fe640 1 -- 172.21.15.5:0/1240619316 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e80b37d0 con 0x7f33e80b2f40 2023-11-24T06:16:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.118+0000 7f33e77fe640 1 -- 172.21.15.5:0/1240619316 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f33e00019f0 con 0x7f33e80b2f40 2023-11-24T06:16:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.118+0000 7f33e77fe640 1 -- 172.21.15.5:0/1240619316 >> v1:172.21.15.5:6789/0 conn(0x7f33e80b2f40 legacy=0x7f33e8158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.119+0000 7f33e77fe640 1 --2- 172.21.15.5:0/1240619316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33d40039c0 0x7f33d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.119+0000 7f33e77fe640 1 --2- 172.21.15.5:0/1240619316 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33d40047c0 0x7f33d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.119+0000 7f33e77fe640 1 --2- 172.21.15.5:0/1240619316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33d4005180 0x7f33d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.119+0000 7f33e77fe640 1 -- 172.21.15.5:0/1240619316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e815aec0 con 0x7f33d40039c0 2023-11-24T06:16:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.119+0000 7f33e77fe640 1 -- 172.21.15.5:0/1240619316 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d400c080 con 0x7f33d4005180 2023-11-24T06:16:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.119+0000 7f33e77fe640 1 -- 172.21.15.5:0/1240619316 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d400c240 con 0x7f33d40047c0 2023-11-24T06:16:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.119+0000 7f33e7fff640 1 --2- 172.21.15.5:0/1240619316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33d40039c0 0x7f33d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.119+0000 7f33ece38640 1 --2- 172.21.15.5:0/1240619316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33d4005180 0x7f33d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.119+0000 7f33e7fff640 1 -- 172.21.15.5:0/1240619316 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33d40047c0 msgr2=0x7f33d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.119+0000 7f33e7fff640 1 --2- 172.21.15.5:0/1240619316 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33d40047c0 0x7f33d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.119+0000 7f33e7fff640 1 -- 172.21.15.5:0/1240619316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33d4005180 msgr2=0x7f33d400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.119+0000 7f33e7fff640 1 --2- 172.21.15.5:0/1240619316 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33d4005180 0x7f33d400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.119+0000 7f33e7fff640 1 -- 172.21.15.5:0/1240619316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d4003520 con 0x7f33d40039c0 2023-11-24T06:16:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.119+0000 7f33ece38640 1 -- 172.21.15.5:0/1240619316 reap_dead start 2023-11-24T06:16:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.120+0000 7f33e7fff640 1 --2- 172.21.15.5:0/1240619316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33d40039c0 0x7f33d4003de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f33d800aac0 tx=0x7f33d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cd4a63b06b217f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.121+0000 7f33e77fe640 1 -- 172.21.15.5:0/1240619316 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8034600 con 0x7f33d40039c0 2023-11-24T06:16:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.121+0000 7f33e77fe640 1 -- 172.21.15.5:0/1240619316 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d8030050 con 0x7f33d40039c0 2023-11-24T06:16:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.121+0000 7f33e77fe640 1 -- 172.21.15.5:0/1240619316 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8034980 con 0x7f33d40039c0 2023-11-24T06:16:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.121+0000 7f33ee89b640 1 -- 172.21.15.5:0/1240619316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33d40039c0 msgr2=0x7f33d4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.121+0000 7f33ee89b640 1 --2- 172.21.15.5:0/1240619316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33d40039c0 0x7f33d4003de0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f33d800aac0 tx=0x7f33d40057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.122+0000 7f33ee89b640 1 -- 172.21.15.5:0/1240619316 shutdown_connections 2023-11-24T06:16:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.122+0000 7f33ee89b640 1 --2- 172.21.15.5:0/1240619316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33d40039c0 0x7f33d4003de0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.122+0000 7f33ee89b640 1 -- 172.21.15.5:0/1240619316 >> 172.21.15.5:0/1240619316 conn(0x7f33e80b0ba0 msgr2=0x7f33e80b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.122+0000 7f33ee89b640 1 -- 172.21.15.5:0/1240619316 shutdown_connections 2023-11-24T06:16:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.122+0000 7f33ee89b640 1 -- 172.21.15.5:0/1240619316 wait complete. 2023-11-24T06:16:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.124+0000 7f33ee89b640 1 Processor -- start 2023-11-24T06:16:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.124+0000 7f33ee89b640 1 -- start start 2023-11-24T06:16:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33ee89b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33d40039c0 0x7f33e809c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33ee89b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33e81f3360 0x7f33e81529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33ee89b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33e8152f30 0x7f33e8154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33ee89b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e81533a0 con 0x7f33e81f3360 2023-11-24T06:16:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33ee89b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e8155d40 con 0x7f33e8152f30 2023-11-24T06:16:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33ee89b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e8155130 con 0x7f33d40039c0 2023-11-24T06:16:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33e81f3360 0x7f33e81529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33dffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33e81f3360 0x7f33e81529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40564/0 (socket says 172.21.15.5:40564) 2023-11-24T06:16:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33dffff640 1 -- 172.21.15.5:0/3025991791 learned_addr learned my addr 172.21.15.5:0/3025991791 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33e7fff640 1 --2- 172.21.15.5:0/3025991791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33d40039c0 0x7f33e809c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33ece38640 1 --2- 172.21.15.5:0/3025991791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33e8152f30 0x7f33e8154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33dffff640 1 -- 172.21.15.5:0/3025991791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33d40039c0 msgr2=0x7f33e809c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33dffff640 1 --2- 172.21.15.5:0/3025991791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33d40039c0 0x7f33e809c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33dffff640 1 -- 172.21.15.5:0/3025991791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33e8152f30 msgr2=0x7f33e8154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33dffff640 1 --2- 172.21.15.5:0/3025991791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33e8152f30 0x7f33e8154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.125+0000 7f33dffff640 1 -- 172.21.15.5:0/3025991791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e8155410 con 0x7f33e81f3360 2023-11-24T06:16:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.126+0000 7f33dffff640 1 --2- 172.21.15.5:0/3025991791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33e81f3360 0x7f33e81529f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f33d000c500 tx=0x7f33d000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.126+0000 7f33e57fa640 1 -- 172.21.15.5:0/3025991791 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d00052d0 con 0x7f33e81f3360 2023-11-24T06:16:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.126+0000 7f33e57fa640 1 -- 172.21.15.5:0/3025991791 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d0005450 con 0x7f33e81f3360 2023-11-24T06:16:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.126+0000 7f33e57fa640 1 -- 172.21.15.5:0/3025991791 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0003830 con 0x7f33e81f3360 2023-11-24T06:16:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.126+0000 7f33ee89b640 1 -- 172.21.15.5:0/3025991791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e81556f0 con 0x7f33e81f3360 2023-11-24T06:16:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.126+0000 7f33ee89b640 1 -- 172.21.15.5:0/3025991791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e809c620 con 0x7f33e81f3360 2023-11-24T06:16:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.128+0000 7f33ee89b640 1 -- 172.21.15.5:0/3025991791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e8155ef0 con 0x7f33e81f3360 2023-11-24T06:16:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.128+0000 7f33e57fa640 1 -- 172.21.15.5:0/3025991791 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33d0003990 con 0x7f33e81f3360 2023-11-24T06:16:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.130+0000 7f33e57fa640 1 --2- 172.21.15.5:0/3025991791 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f33c806ae30 0x7f33c806d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.130+0000 7f33e57fa640 1 -- 172.21.15.5:0/3025991791 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33d0088790 con 0x7f33e81f3360 2023-11-24T06:16:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.130+0000 7f33e7fff640 1 --2- 172.21.15.5:0/3025991791 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f33c806ae30 0x7f33c806d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.131+0000 7f33e7fff640 1 --2- 172.21.15.5:0/3025991791 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f33c806ae30 0x7f33c806d2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33d8037000 tx=0x7f33d8032000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.135+0000 7f33e57fa640 1 -- 172.21.15.5:0/3025991791 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f33e8155ef0 con 0x7f33e81f3360 2023-11-24T06:16:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.327+0000 7f33ee89b640 1 -- 172.21.15.5:0/3025991791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33e8155eb0 con 0x7f33e81f3360 2023-11-24T06:16:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.328+0000 7f33e57fa640 1 -- 172.21.15.5:0/3025991791 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f33d0057370 con 0x7f33e81f3360 2023-11-24T06:16:32.331 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:32.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.331+0000 7f33ee89b640 1 -- 172.21.15.5:0/3025991791 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f33c806ae30 msgr2=0x7f33c806d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:32.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.331+0000 7f33ee89b640 1 --2- 172.21.15.5:0/3025991791 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f33c806ae30 0x7f33c806d2f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f33d8037000 tx=0x7f33d8032000 comp rx=0 tx=0).stop 2023-11-24T06:16:32.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.331+0000 7f33ee89b640 1 -- 172.21.15.5:0/3025991791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33e81f3360 msgr2=0x7f33e81529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.331+0000 7f33ee89b640 1 --2- 172.21.15.5:0/3025991791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33e81f3360 0x7f33e81529f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f33d000c500 tx=0x7f33d000c9c0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.331+0000 7f33ee89b640 1 -- 172.21.15.5:0/3025991791 shutdown_connections 2023-11-24T06:16:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.331+0000 7f33ee89b640 1 --2- 172.21.15.5:0/3025991791 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f33c806ae30 0x7f33c806d2f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.331+0000 7f33ee89b640 1 --2- 172.21.15.5:0/3025991791 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f33e8152f30 0x7f33e8154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.331+0000 7f33ee89b640 1 --2- 172.21.15.5:0/3025991791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33e81f3360 0x7f33e81529f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.331+0000 7f33ee89b640 1 --2- 172.21.15.5:0/3025991791 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f33d40039c0 0x7f33e809c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.331+0000 7f33ee89b640 1 -- 172.21.15.5:0/3025991791 >> 172.21.15.5:0/3025991791 conn(0x7f33e80b0ba0 msgr2=0x7f33e81536f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.332+0000 7f33ee89b640 1 -- 172.21.15.5:0/3025991791 shutdown_connections 2023-11-24T06:16:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.332+0000 7f33ee89b640 1 -- 172.21.15.5:0/3025991791 wait complete. 2023-11-24T06:16:32.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.577+0000 7fcb09d8a640 1 Processor -- start 2023-11-24T06:16:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.577+0000 7fcb09d8a640 1 -- start start 2023-11-24T06:16:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.579+0000 7fcb09d8a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb0415f270 0x7fcb040cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.579+0000 7fcb09d8a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0415f930 con 0x7fcb0406d9e0 2023-11-24T06:16:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.579+0000 7fcb09d8a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb040b3b10 con 0x7fcb04144770 2023-11-24T06:16:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.579+0000 7fcb09d8a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040b43b0 con 0x7fcb0415f270 2023-11-24T06:16:32.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.580+0000 7fcb09589640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcb0406d9e0 0x7fcb040cd9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35568/0 (socket says 172.21.15.5:35568) 2023-11-24T06:16:32.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.580+0000 7fcb09589640 1 -- 172.21.15.5:0/806256720 learned_addr learned my addr 172.21.15.5:0/806256720 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:32.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.580+0000 7fcb03fff640 1 --2- 172.21.15.5:0/806256720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb0415f270 0x7fcb040cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.580+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585248420 0 0) 0x7fcb0415f930 con 0x7fcb0406d9e0 2023-11-24T06:16:32.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.580+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaf4003520 con 0x7fcb0406d9e0 2023-11-24T06:16:32.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046579344 0 0) 0x7fcb040b3b10 con 0x7fcb04144770 2023-11-24T06:16:32.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb0415f930 con 0x7fcb04144770 2023-11-24T06:16:32.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2779940725 0 0) 0x7fcaf4003520 con 0x7fcb0406d9e0 2023-11-24T06:16:32.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb0415f270 msgr2=0x7fcb040cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:32.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 --2- 172.21.15.5:0/806256720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb0415f270 0x7fcb040cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 >> v1:172.21.15.6:6789/0 conn(0x7fcb04144770 legacy=0x7fcb0415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040b3df0 con 0x7fcb0406d9e0 2023-11-24T06:16:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fcafc0019f0 con 0x7fcb0406d9e0 2023-11-24T06:16:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb03fff640 1 --2- 172.21.15.5:0/806256720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb0415f270 0x7fcb040cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:16:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 >> v1:172.21.15.5:6789/0 conn(0x7fcb0406d9e0 legacy=0x7fcb040cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 --2- 172.21.15.5:0/806256720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb0415f270 0x7fcaf4003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 --2- 172.21.15.5:0/806256720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaf40040e0 0x7fcaf40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 --2- 172.21.15.5:0/806256720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4004d00 0x7fcaf4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040b43b0 con 0x7fcaf40040e0 2023-11-24T06:16:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf400bac0 con 0x7fcaf4004d00 2023-11-24T06:16:32.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf400bc80 con 0x7fcb0415f270 2023-11-24T06:16:32.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.581+0000 7fcb03fff640 1 --2- 172.21.15.5:0/806256720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaf40040e0 0x7fcaf40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.582+0000 7fcb08d88640 1 --2- 172.21.15.5:0/806256720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb0415f270 0x7fcaf4003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.582+0000 7fcb09589640 1 --2- 172.21.15.5:0/806256720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4004d00 0x7fcaf4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.582+0000 7fcb03fff640 1 -- 172.21.15.5:0/806256720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb0415f270 msgr2=0x7fcaf4003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:32.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.582+0000 7fcb03fff640 1 --2- 172.21.15.5:0/806256720 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb0415f270 0x7fcaf4003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.582+0000 7fcb03fff640 1 -- 172.21.15.5:0/806256720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4004d00 msgr2=0x7fcaf4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:32.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.582+0000 7fcb03fff640 1 --2- 172.21.15.5:0/806256720 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcaf4004d00 0x7fcaf4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.582+0000 7fcb03fff640 1 -- 172.21.15.5:0/806256720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf0009400 con 0x7fcaf40040e0 2023-11-24T06:16:32.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.583+0000 7fcb03fff640 1 --2- 172.21.15.5:0/806256720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaf40040e0 0x7fcaf40047c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009c70 tx=0x7fcaf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5881b235f3b086fe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:32.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.584+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf002c860 con 0x7fcaf40040e0 2023-11-24T06:16:32.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.584+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf002c9c0 con 0x7fcaf40040e0 2023-11-24T06:16:32.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.584+0000 7fcb037fe640 1 -- 172.21.15.5:0/806256720 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf002cd70 con 0x7fcaf40040e0 2023-11-24T06:16:32.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.584+0000 7fcb09d8a640 1 -- 172.21.15.5:0/806256720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaf40040e0 msgr2=0x7fcaf40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:32.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.584+0000 7fcb09d8a640 1 --2- 172.21.15.5:0/806256720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaf40040e0 0x7fcaf40047c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009c70 tx=0x7fcaf40057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.584+0000 7fcb09589640 1 -- 172.21.15.5:0/806256720 reap_dead start 2023-11-24T06:16:32.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.584+0000 7fcb09d8a640 1 -- 172.21.15.5:0/806256720 shutdown_connections 2023-11-24T06:16:32.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.584+0000 7fcb09d8a640 1 -- 172.21.15.5:0/806256720 >> 172.21.15.5:0/806256720 conn(0x7fcb040cec10 msgr2=0x7fcb040cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:32.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.584+0000 7fcb09d8a640 1 -- 172.21.15.5:0/806256720 shutdown_connections 2023-11-24T06:16:32.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.584+0000 7fcb09d8a640 1 -- 172.21.15.5:0/806256720 wait complete. 2023-11-24T06:16:32.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.586+0000 7fcb09d8a640 1 Processor -- start 2023-11-24T06:16:32.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.587+0000 7fcb09d8a640 1 -- start start 2023-11-24T06:16:32.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.587+0000 7fcb09d8a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaf40040e0 0x7fcb0413c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb09d8a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb0413b260 0x7fcb0413b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb09d8a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0413bc20 0x7fcb0414c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb09d8a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0413c0a0 con 0x7fcaf40040e0 2023-11-24T06:16:32.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb09d8a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0414a800 con 0x7fcb0413bc20 2023-11-24T06:16:32.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb09d8a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040a8b20 con 0x7fcb0413b260 2023-11-24T06:16:32.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb08d88640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaf40040e0 0x7fcb0413c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb08d88640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaf40040e0 0x7fcb0413c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40588/0 (socket says 172.21.15.5:40588) 2023-11-24T06:16:32.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb08d88640 1 -- 172.21.15.5:0/330452119 learned_addr learned my addr 172.21.15.5:0/330452119 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:32.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb03fff640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb0413b260 0x7fcb0413b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb09589640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0413bc20 0x7fcb0414c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb08d88640 1 -- 172.21.15.5:0/330452119 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb0413b260 msgr2=0x7fcb0413b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb08d88640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb0413b260 0x7fcb0413b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb08d88640 1 -- 172.21.15.5:0/330452119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0413bc20 msgr2=0x7fcb0414c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb08d88640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0413bc20 0x7fcb0414c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb08d88640 1 -- 172.21.15.5:0/330452119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040a8840 con 0x7fcaf40040e0 2023-11-24T06:16:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.588+0000 7fcb09589640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0413bc20 0x7fcb0414c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.589+0000 7fcb08d88640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaf40040e0 0x7fcb0413c710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800e7b0 tx=0x7fcaf800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.589+0000 7fcb017fa640 1 -- 172.21.15.5:0/330452119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf800ee60 con 0x7fcaf40040e0 2023-11-24T06:16:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.589+0000 7fcb017fa640 1 -- 172.21.15.5:0/330452119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf80052e0 con 0x7fcaf40040e0 2023-11-24T06:16:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.589+0000 7fcb017fa640 1 -- 172.21.15.5:0/330452119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf800c6c0 con 0x7fcaf40040e0 2023-11-24T06:16:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.589+0000 7fcb09d8a640 1 -- 172.21.15.5:0/330452119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0414aaa0 con 0x7fcaf40040e0 2023-11-24T06:16:32.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.589+0000 7fcb09d8a640 1 -- 172.21.15.5:0/330452119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb040a8460 con 0x7fcaf40040e0 2023-11-24T06:16:32.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.591+0000 7fcb09d8a640 1 -- 172.21.15.5:0/330452119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0414a660 con 0x7fcaf40040e0 2023-11-24T06:16:32.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.591+0000 7fcb017fa640 1 -- 172.21.15.5:0/330452119 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcaf800c820 con 0x7fcaf40040e0 2023-11-24T06:16:32.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.592+0000 7fcb017fa640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fcae806ae70 0x7fcae806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:32.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.593+0000 7fcb017fa640 1 -- 172.21.15.5:0/330452119 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcaf8089580 con 0x7fcaf40040e0 2023-11-24T06:16:32.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.593+0000 7fcb03fff640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fcae806ae70 0x7fcae806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:32.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.594+0000 7fcb03fff640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fcae806ae70 0x7fcae806d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0036000 tx=0x7fcaf0030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:32.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.597+0000 7fcb017fa640 1 -- 172.21.15.5:0/330452119 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb0414a660 con 0x7fcaf40040e0 2023-11-24T06:16:32.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.793+0000 7fcb09d8a640 1 -- 172.21.15.5:0/330452119 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb0414afc0 con 0x7fcaf40040e0 2023-11-24T06:16:32.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.794+0000 7fcb017fa640 1 -- 172.21.15.5:0/330452119 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb0414afc0 con 0x7fcaf40040e0 2023-11-24T06:16:32.797 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:32.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.797+0000 7fcb09d8a640 1 -- 172.21.15.5:0/330452119 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fcae806ae70 msgr2=0x7fcae806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:32.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.797+0000 7fcb09d8a640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fcae806ae70 0x7fcae806d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0036000 tx=0x7fcaf0030000 comp rx=0 tx=0).stop 2023-11-24T06:16:32.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.797+0000 7fcb09d8a640 1 -- 172.21.15.5:0/330452119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaf40040e0 msgr2=0x7fcb0413c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:32.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.797+0000 7fcb09d8a640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaf40040e0 0x7fcb0413c710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800e7b0 tx=0x7fcaf800ec70 comp rx=0 tx=0).stop 2023-11-24T06:16:32.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.797+0000 7fcb09d8a640 1 -- 172.21.15.5:0/330452119 shutdown_connections 2023-11-24T06:16:32.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.797+0000 7fcb09d8a640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fcae806ae70 0x7fcae806d330 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.797+0000 7fcb09d8a640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcb0413b260 0x7fcb0413b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.797+0000 7fcb09d8a640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcb0413bc20 0x7fcb0414c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.797+0000 7fcb09d8a640 1 --2- 172.21.15.5:0/330452119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcaf40040e0 0x7fcb0413c710 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:32.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.797+0000 7fcb09d8a640 1 -- 172.21.15.5:0/330452119 >> 172.21.15.5:0/330452119 conn(0x7fcb040cec10 msgr2=0x7fcb040cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:32.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.797+0000 7fcb09d8a640 1 -- 172.21.15.5:0/330452119 shutdown_connections 2023-11-24T06:16:32.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:32.797+0000 7fcb09d8a640 1 -- 172.21.15.5:0/330452119 wait complete. 2023-11-24T06:16:32.885 INFO:tasks.cephfs_test_runner:Starting test: test_clone_failure_status_failed (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) ensure failure status is shown when clone is in failed state and validate the reason 2023-11-24T06:16:32.886 INFO:tasks.cephfs_test_runner:test_clone_failure_status_failed (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2023-11-24T06:16:32.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:33.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.118+0000 7ffaaa42e640 1 Processor -- start 2023-11-24T06:16:33.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.119+0000 7ffaaa42e640 1 -- start start 2023-11-24T06:16:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.120+0000 7ffaaa42e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ffaa415d370 0x7ffaa40d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.120+0000 7ffaaa42e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ffaa40d00d0 0x7ffaa40b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:33.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.120+0000 7ffaaa42e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa40afd30 con 0x7ffaa40b1f00 2023-11-24T06:16:33.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.120+0000 7ffaaa42e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa40d0c30 con 0x7ffaa40d00d0 2023-11-24T06:16:33.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.120+0000 7ffaaa42e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa40cef60 con 0x7ffaa415d370 2023-11-24T06:16:33.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.120+0000 7ffaa8c2b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ffaa40b1f00 0x7ffaa40cf9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56282/0 (socket says 172.21.15.5:56282) 2023-11-24T06:16:33.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.120+0000 7ffaa8c2b640 1 -- 172.21.15.5:0/689196449 learned_addr learned my addr 172.21.15.5:0/689196449 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:33.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.121+0000 7ffaa9c2d640 1 --2- 172.21.15.5:0/689196449 >> v2:172.21.15.5:3300/0 conn(0x7ffaa40d00d0 0x7ffaa40b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.121+0000 7ffaa942c640 1 --2- 172.21.15.5:0/689196449 >> v2:172.21.15.6:3300/0 conn(0x7ffaa415d370 0x7ffaa40d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.121+0000 7ffa9bfff640 1 -- 172.21.15.5:0/689196449 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816647666 0 0) 0x7ffaa40afd30 con 0x7ffaa40b1f00 2023-11-24T06:16:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.121+0000 7ffa9bfff640 1 -- 172.21.15.5:0/689196449 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa90003520 con 0x7ffaa40b1f00 2023-11-24T06:16:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.121+0000 7ffaa9c2d640 1 -- 172.21.15.5:0/689196449 >> v2:172.21.15.6:3300/0 conn(0x7ffaa415d370 msgr2=0x7ffaa40d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.121+0000 7ffaa9c2d640 1 --2- 172.21.15.5:0/689196449 >> v2:172.21.15.6:3300/0 conn(0x7ffaa415d370 0x7ffaa40d1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.121+0000 7ffaa9c2d640 1 -- 172.21.15.5:0/689196449 >> v1:172.21.15.5:6789/0 conn(0x7ffaa40b1f00 legacy=0x7ffaa40cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.121+0000 7ffaa9c2d640 1 -- 172.21.15.5:0/689196449 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa40d0ef0 con 0x7ffaa40d00d0 2023-11-24T06:16:33.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.121+0000 7ffaa942c640 1 --2- 172.21.15.5:0/689196449 >> v2:172.21.15.6:3300/0 conn(0x7ffaa415d370 0x7ffaa40d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:16:33.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.122+0000 7ffaa9c2d640 1 --2- 172.21.15.5:0/689196449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40d00d0 0x7ffaa40b04a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffaa408b8b0 tx=0x7ffa9c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7aff5a72395891f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.123+0000 7ffa9bfff640 1 -- 172.21.15.5:0/689196449 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c02cbf0 con 0x7ffaa40d00d0 2023-11-24T06:16:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.123+0000 7ffa9bfff640 1 -- 172.21.15.5:0/689196449 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c02cd50 con 0x7ffaa40d00d0 2023-11-24T06:16:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.123+0000 7ffa9bfff640 1 -- 172.21.15.5:0/689196449 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c034640 con 0x7ffaa40d00d0 2023-11-24T06:16:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.123+0000 7ffaaa42e640 1 -- 172.21.15.5:0/689196449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40d00d0 msgr2=0x7ffaa40b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.123+0000 7ffaaa42e640 1 --2- 172.21.15.5:0/689196449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40d00d0 0x7ffaa40b04a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffaa408b8b0 tx=0x7ffa9c02be40 comp rx=0 tx=0).stop 2023-11-24T06:16:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.123+0000 7ffaaa42e640 1 -- 172.21.15.5:0/689196449 shutdown_connections 2023-11-24T06:16:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.123+0000 7ffaaa42e640 1 --2- 172.21.15.5:0/689196449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40d00d0 0x7ffaa40b04a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.123+0000 7ffaaa42e640 1 --2- 172.21.15.5:0/689196449 >> v2:172.21.15.6:3300/0 conn(0x7ffaa415d370 0x7ffaa40d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.124+0000 7ffaaa42e640 1 -- 172.21.15.5:0/689196449 >> 172.21.15.5:0/689196449 conn(0x7ffaa409c3d0 msgr2=0x7ffaa40d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.124+0000 7ffaaa42e640 1 -- 172.21.15.5:0/689196449 shutdown_connections 2023-11-24T06:16:33.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.124+0000 7ffaaa42e640 1 -- 172.21.15.5:0/689196449 wait complete. 2023-11-24T06:16:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.125+0000 7ffaaa42e640 1 Processor -- start 2023-11-24T06:16:33.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.126+0000 7ffaaa42e640 1 -- start start 2023-11-24T06:16:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.126+0000 7ffaaa42e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40b1f00 0x7ffaa40b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.126+0000 7ffaaa42e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffaa40d00d0 0x7ffaa41f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.127+0000 7ffaaa42e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffaa415d370 0x7ffaa415a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.127+0000 7ffaaa42e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa40b3350 con 0x7ffaa40b1f00 2023-11-24T06:16:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.127+0000 7ffaaa42e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa40cbe50 con 0x7ffaa415d370 2023-11-24T06:16:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.127+0000 7ffaaa42e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa40cc130 con 0x7ffaa40d00d0 2023-11-24T06:16:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.127+0000 7ffaa942c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40b1f00 0x7ffaa40b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.127+0000 7ffaa942c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40b1f00 0x7ffaa40b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50510/0 (socket says 172.21.15.5:50510) 2023-11-24T06:16:33.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.127+0000 7ffaa942c640 1 -- 172.21.15.5:0/266913271 learned_addr learned my addr 172.21.15.5:0/266913271 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:33.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.127+0000 7ffaa942c640 1 -- 172.21.15.5:0/266913271 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffaa40d00d0 msgr2=0x7ffaa41f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:33.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.127+0000 7ffaa942c640 1 --2- 172.21.15.5:0/266913271 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffaa40d00d0 0x7ffaa41f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.127+0000 7ffaa9c2d640 1 --2- 172.21.15.5:0/266913271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffaa415d370 0x7ffaa415a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:33.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.127+0000 7ffaa942c640 1 -- 172.21.15.5:0/266913271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffaa415d370 msgr2=0x7ffaa415a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:33.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.127+0000 7ffaa942c640 1 --2- 172.21.15.5:0/266913271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffaa415d370 0x7ffaa415a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.127+0000 7ffaa942c640 1 -- 172.21.15.5:0/266913271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c009a70 con 0x7ffaa40b1f00 2023-11-24T06:16:33.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.128+0000 7ffaa942c640 1 --2- 172.21.15.5:0/266913271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40b1f00 0x7ffaa40b2e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000e730 tx=0x7ffaa000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:33.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.128+0000 7ffa99ffb640 1 -- 172.21.15.5:0/266913271 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0004230 con 0x7ffaa40b1f00 2023-11-24T06:16:33.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.128+0000 7ffa99ffb640 1 -- 172.21.15.5:0/266913271 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa0004390 con 0x7ffaa40b1f00 2023-11-24T06:16:33.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.128+0000 7ffaaa42e640 1 -- 172.21.15.5:0/266913271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa40cc410 con 0x7ffaa40b1f00 2023-11-24T06:16:33.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.128+0000 7ffa99ffb640 1 -- 172.21.15.5:0/266913271 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa000c8f0 con 0x7ffaa40b1f00 2023-11-24T06:16:33.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.128+0000 7ffaaa42e640 1 -- 172.21.15.5:0/266913271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa40cc9d0 con 0x7ffaa40b1f00 2023-11-24T06:16:33.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.129+0000 7ffaaa42e640 1 -- 172.21.15.5:0/266913271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa40ccb90 con 0x7ffaa40b1f00 2023-11-24T06:16:33.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.130+0000 7ffa99ffb640 1 -- 172.21.15.5:0/266913271 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffaa000ca50 con 0x7ffaa40b1f00 2023-11-24T06:16:33.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.131+0000 7ffa99ffb640 1 --2- 172.21.15.5:0/266913271 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ffa8806ae40 0x7ffa8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:33.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.131+0000 7ffa99ffb640 1 -- 172.21.15.5:0/266913271 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ffaa0089340 con 0x7ffaa40b1f00 2023-11-24T06:16:33.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.132+0000 7ffaa8c2b640 1 --2- 172.21.15.5:0/266913271 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ffa8806ae40 0x7ffa8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:33.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.132+0000 7ffaa8c2b640 1 --2- 172.21.15.5:0/266913271 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ffa8806ae40 0x7ffa8806d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffa94000f60 tx=0x7ffa940078f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:33.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.135+0000 7ffa99ffb640 1 -- 172.21.15.5:0/266913271 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffaa00542c0 con 0x7ffaa40b1f00 2023-11-24T06:16:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.329+0000 7ffaaa42e640 1 -- 172.21.15.5:0/266913271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffaa404e840 con 0x7ffaa40b1f00 2023-11-24T06:16:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.330+0000 7ffa99ffb640 1 -- 172.21.15.5:0/266913271 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16658 (secure 0 0 0) 0x7ffaa00578e0 con 0x7ffaa40b1f00 2023-11-24T06:16:33.333 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:33.334 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:16:32.946913+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.333+0000 7ffaaa42e640 1 -- 172.21.15.5:0/266913271 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ffa8806ae40 msgr2=0x7ffa8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.333+0000 7ffaaa42e640 1 --2- 172.21.15.5:0/266913271 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ffa8806ae40 0x7ffa8806d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffa94000f60 tx=0x7ffa940078f0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.333+0000 7ffaaa42e640 1 -- 172.21.15.5:0/266913271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40b1f00 msgr2=0x7ffaa40b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.333+0000 7ffaaa42e640 1 --2- 172.21.15.5:0/266913271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40b1f00 0x7ffaa40b2e10 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffaa000e730 tx=0x7ffaa000ebf0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.333+0000 7ffaaa42e640 1 -- 172.21.15.5:0/266913271 shutdown_connections 2023-11-24T06:16:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.333+0000 7ffaaa42e640 1 --2- 172.21.15.5:0/266913271 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ffa8806ae40 0x7ffa8806d300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.333+0000 7ffaaa42e640 1 --2- 172.21.15.5:0/266913271 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ffaa40d00d0 0x7ffaa41f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.333+0000 7ffaaa42e640 1 --2- 172.21.15.5:0/266913271 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ffaa415d370 0x7ffaa415a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.333+0000 7ffaaa42e640 1 --2- 172.21.15.5:0/266913271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffaa40b1f00 0x7ffaa40b2e10 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.333+0000 7ffaaa42e640 1 -- 172.21.15.5:0/266913271 >> 172.21.15.5:0/266913271 conn(0x7ffaa409c3d0 msgr2=0x7ffaa40b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.333+0000 7ffaaa42e640 1 -- 172.21.15.5:0/266913271 shutdown_connections 2023-11-24T06:16:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.333+0000 7ffaaa42e640 1 -- 172.21.15.5:0/266913271 wait complete. 2023-11-24T06:16:33.355 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:33.356 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:33.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.585+0000 7fd0dafc9640 1 Processor -- start 2023-11-24T06:16:33.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.586+0000 7fd0dafc9640 1 -- start start 2023-11-24T06:16:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.587+0000 7fd0dafc9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0d40ce6b0 0x7fd0d40cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.587+0000 7fd0dafc9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d40cd2b0 con 0x7fd0d40cf380 2023-11-24T06:16:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.587+0000 7fd0dafc9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d40d02e0 con 0x7fd0d4158f40 2023-11-24T06:16:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.587+0000 7fd0dafc9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d40cd4e0 con 0x7fd0d40ce6b0 2023-11-24T06:16:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.588+0000 7fd0d9566640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0d40ce6b0 0x7fd0d40cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.588+0000 7fd0d9566640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0d40ce6b0 0x7fd0d40cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50526/0 (socket says 172.21.15.5:50526) 2023-11-24T06:16:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.588+0000 7fd0d9566640 1 -- 172.21.15.5:0/3615115576 learned_addr learned my addr 172.21.15.5:0/3615115576 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.588+0000 7fd0cbfff640 1 --1- 172.21.15.5:0/3615115576 >> v1:172.21.15.5:6789/0 conn(0x7fd0d40cf380 0x7fd0d415b830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56296/0 (socket says 172.21.15.5:56296) 2023-11-24T06:16:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.588+0000 7fd0d9566640 1 -- 172.21.15.5:0/3615115576 >> v1:172.21.15.6:6789/0 conn(0x7fd0d4158f40 legacy=0x7fd0d40cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.588+0000 7fd0d9566640 1 -- 172.21.15.5:0/3615115576 >> v1:172.21.15.5:6789/0 conn(0x7fd0d40cf380 legacy=0x7fd0d415b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.588+0000 7fd0d9566640 1 -- 172.21.15.5:0/3615115576 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d40cd7c0 con 0x7fd0d40ce6b0 2023-11-24T06:16:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.590+0000 7fd0d9566640 1 --2- 172.21.15.5:0/3615115576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d40ce6b0 0x7fd0d40cb9d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009e50 tx=0x7fd0cc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bfd8122b060223 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.590+0000 7fd0cb7fe640 1 -- 172.21.15.5:0/3615115576 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc02cb20 con 0x7fd0d40ce6b0 2023-11-24T06:16:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.590+0000 7fd0cb7fe640 1 -- 172.21.15.5:0/3615115576 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc02cc80 con 0x7fd0d40ce6b0 2023-11-24T06:16:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.590+0000 7fd0cb7fe640 1 -- 172.21.15.5:0/3615115576 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0343f0 con 0x7fd0d40ce6b0 2023-11-24T06:16:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.590+0000 7fd0dafc9640 1 -- 172.21.15.5:0/3615115576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d40ce6b0 msgr2=0x7fd0d40cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.590+0000 7fd0dafc9640 1 --2- 172.21.15.5:0/3615115576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d40ce6b0 0x7fd0d40cb9d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc009e50 tx=0x7fd0cc02bf80 comp rx=0 tx=0).stop 2023-11-24T06:16:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.591+0000 7fd0dafc9640 1 -- 172.21.15.5:0/3615115576 shutdown_connections 2023-11-24T06:16:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.591+0000 7fd0dafc9640 1 --2- 172.21.15.5:0/3615115576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d40ce6b0 0x7fd0d40cb9d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.591+0000 7fd0dafc9640 1 -- 172.21.15.5:0/3615115576 >> 172.21.15.5:0/3615115576 conn(0x7fd0d40cdb20 msgr2=0x7fd0d40cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.591+0000 7fd0dafc9640 1 -- 172.21.15.5:0/3615115576 shutdown_connections 2023-11-24T06:16:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.591+0000 7fd0dafc9640 1 -- 172.21.15.5:0/3615115576 wait complete. 2023-11-24T06:16:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.592+0000 7fd0dafc9640 1 Processor -- start 2023-11-24T06:16:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.593+0000 7fd0dafc9640 1 -- start start 2023-11-24T06:16:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.594+0000 7fd0dafc9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0d40ce6b0 0x7fd0d415e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.594+0000 7fd0dafc9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d40cf380 0x7fd0d41f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.594+0000 7fd0dafc9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0d4158f40 0x7fd0d41f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.594+0000 7fd0dafc9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d40c4f10 con 0x7fd0d40cf380 2023-11-24T06:16:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.594+0000 7fd0dafc9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d40ca890 con 0x7fd0d4158f40 2023-11-24T06:16:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.594+0000 7fd0dafc9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d40cab70 con 0x7fd0d40ce6b0 2023-11-24T06:16:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.594+0000 7fd0cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d40cf380 0x7fd0d41f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.594+0000 7fd0cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d40cf380 0x7fd0d41f4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50534/0 (socket says 172.21.15.5:50534) 2023-11-24T06:16:33.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.594+0000 7fd0cbfff640 1 -- 172.21.15.5:0/1091993859 learned_addr learned my addr 172.21.15.5:0/1091993859 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.594+0000 7fd0d8d65640 1 --2- 172.21.15.5:0/1091993859 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0d40ce6b0 0x7fd0d415e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.594+0000 7fd0d9566640 1 --2- 172.21.15.5:0/1091993859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0d4158f40 0x7fd0d41f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.595+0000 7fd0cbfff640 1 -- 172.21.15.5:0/1091993859 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0d40ce6b0 msgr2=0x7fd0d415e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.595+0000 7fd0cbfff640 1 --2- 172.21.15.5:0/1091993859 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0d40ce6b0 0x7fd0d415e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.595+0000 7fd0cbfff640 1 -- 172.21.15.5:0/1091993859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0d4158f40 msgr2=0x7fd0d41f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.595+0000 7fd0cbfff640 1 --2- 172.21.15.5:0/1091993859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0d4158f40 0x7fd0d41f5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.595+0000 7fd0cbfff640 1 -- 172.21.15.5:0/1091993859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc009b30 con 0x7fd0d40cf380 2023-11-24T06:16:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.595+0000 7fd0cbfff640 1 --2- 172.21.15.5:0/1091993859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d40cf380 0x7fd0d41f4d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400c4d0 tx=0x7fd0c400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.595+0000 7fd0c97fa640 1 -- 172.21.15.5:0/1091993859 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4007480 con 0x7fd0d40cf380 2023-11-24T06:16:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.595+0000 7fd0c97fa640 1 -- 172.21.15.5:0/1091993859 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c40075e0 con 0x7fd0d40cf380 2023-11-24T06:16:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.595+0000 7fd0c97fa640 1 -- 172.21.15.5:0/1091993859 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c40049e0 con 0x7fd0d40cf380 2023-11-24T06:16:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.595+0000 7fd0dafc9640 1 -- 172.21.15.5:0/1091993859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d40cae50 con 0x7fd0d40cf380 2023-11-24T06:16:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.596+0000 7fd0dafc9640 1 -- 172.21.15.5:0/1091993859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d415e560 con 0x7fd0d40cf380 2023-11-24T06:16:33.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.597+0000 7fd0dafc9640 1 -- 172.21.15.5:0/1091993859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d40c50c0 con 0x7fd0d40cf380 2023-11-24T06:16:33.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.598+0000 7fd0c97fa640 1 -- 172.21.15.5:0/1091993859 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd0c4004b40 con 0x7fd0d40cf380 2023-11-24T06:16:33.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.599+0000 7fd0c97fa640 1 --2- 172.21.15.5:0/1091993859 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd0c006ae40 0x7fd0c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:33.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.599+0000 7fd0c97fa640 1 -- 172.21.15.5:0/1091993859 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd0c40886e0 con 0x7fd0d40cf380 2023-11-24T06:16:33.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.599+0000 7fd0d8d65640 1 --2- 172.21.15.5:0/1091993859 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd0c006ae40 0x7fd0c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:33.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.600+0000 7fd0d8d65640 1 --2- 172.21.15.5:0/1091993859 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd0c006ae40 0x7fd0c006d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0006fd0 tx=0x7fd0d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:33.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.603+0000 7fd0c97fa640 1 -- 172.21.15.5:0/1091993859 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0d40c50c0 con 0x7fd0d40cf380 2023-11-24T06:16:33.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.797+0000 7fd0dafc9640 1 -- 172.21.15.5:0/1091993859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0d406f5e0 con 0x7fd0d40cf380 2023-11-24T06:16:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.798+0000 7fd0c97fa640 1 -- 172.21.15.5:0/1091993859 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd0c40537a0 con 0x7fd0d40cf380 2023-11-24T06:16:33.802 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.801+0000 7fd0dafc9640 1 -- 172.21.15.5:0/1091993859 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd0c006ae40 msgr2=0x7fd0c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.801+0000 7fd0dafc9640 1 --2- 172.21.15.5:0/1091993859 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd0c006ae40 0x7fd0c006d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0006fd0 tx=0x7fd0d0008040 comp rx=0 tx=0).stop 2023-11-24T06:16:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.801+0000 7fd0dafc9640 1 -- 172.21.15.5:0/1091993859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d40cf380 msgr2=0x7fd0d41f4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.801+0000 7fd0dafc9640 1 --2- 172.21.15.5:0/1091993859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d40cf380 0x7fd0d41f4d10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400c4d0 tx=0x7fd0c400c990 comp rx=0 tx=0).stop 2023-11-24T06:16:33.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.802+0000 7fd0dafc9640 1 -- 172.21.15.5:0/1091993859 shutdown_connections 2023-11-24T06:16:33.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.802+0000 7fd0dafc9640 1 --2- 172.21.15.5:0/1091993859 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fd0c006ae40 0x7fd0c006d300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.802+0000 7fd0dafc9640 1 --2- 172.21.15.5:0/1091993859 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fd0d4158f40 0x7fd0d41f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.802+0000 7fd0dafc9640 1 --2- 172.21.15.5:0/1091993859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0d40cf380 0x7fd0d41f4d10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.802+0000 7fd0dafc9640 1 --2- 172.21.15.5:0/1091993859 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fd0d40ce6b0 0x7fd0d415e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:33.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.802+0000 7fd0dafc9640 1 -- 172.21.15.5:0/1091993859 >> 172.21.15.5:0/1091993859 conn(0x7fd0d40cdb20 msgr2=0x7fd0d40b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:33.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.802+0000 7fd0dafc9640 1 -- 172.21.15.5:0/1091993859 shutdown_connections 2023-11-24T06:16:33.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:33.802+0000 7fd0dafc9640 1 -- 172.21.15.5:0/1091993859 wait complete. 2023-11-24T06:16:33.824 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:34.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.064+0000 7f4bba1c9640 1 Processor -- start 2023-11-24T06:16:34.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.064+0000 7f4bba1c9640 1 -- start start 2023-11-24T06:16:34.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.066+0000 7f4bba1c9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bb4162600 0x7f4bb4162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:34.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.066+0000 7f4bba1c9640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4bb406c3c0 0x7f4bb406c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:34.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.066+0000 7f4bba1c9640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb4162fa0 con 0x7f4bb4162220 2023-11-24T06:16:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.066+0000 7f4bba1c9640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb406cd90 con 0x7f4bb406c3c0 2023-11-24T06:16:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.066+0000 7f4bba1c9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb40706a0 con 0x7f4bb4162600 2023-11-24T06:16:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.067+0000 7f4bb2ffd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bb4162600 0x7f4bb4162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.067+0000 7f4bb37fe640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f4bb4162220 0x7f4bb40b1ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.5:55194/0 (socket says 172.21.15.5:55194) 2023-11-24T06:16:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.067+0000 7f4bb37fe640 1 -- 172.21.15.5:0/1956925389 learned_addr learned my addr 172.21.15.5:0/1956925389 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:34.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.067+0000 7f4bb3fff640 1 --2- 172.21.15.5:0/1956925389 >> v2:172.21.15.6:3300/0 conn(0x7f4bb406c3c0 0x7f4bb406c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.067+0000 7f4bb27fc640 1 -- 172.21.15.5:0/1956925389 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106668739 0 0) 0x7f4bb4162fa0 con 0x7f4bb4162220 2023-11-24T06:16:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.067+0000 7f4bb27fc640 1 -- 172.21.15.5:0/1956925389 --> v1:172.21.15.6:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ba0003520 con 0x7f4bb4162220 2023-11-24T06:16:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.068+0000 7f4bb2ffd640 1 -- 172.21.15.5:0/1956925389 >> v2:172.21.15.6:3300/0 conn(0x7f4bb406c3c0 msgr2=0x7f4bb406c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.068+0000 7f4bb2ffd640 1 --2- 172.21.15.5:0/1956925389 >> v2:172.21.15.6:3300/0 conn(0x7f4bb406c3c0 0x7f4bb406c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.068+0000 7f4bb2ffd640 1 -- 172.21.15.5:0/1956925389 >> v1:172.21.15.6:6789/0 conn(0x7f4bb4162220 legacy=0x7f4bb40b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.068+0000 7f4bb2ffd640 1 -- 172.21.15.5:0/1956925389 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb4070c60 con 0x7f4bb4162600 2023-11-24T06:16:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.068+0000 7f4bb3fff640 1 --2- 172.21.15.5:0/1956925389 >> v2:172.21.15.6:3300/0 conn(0x7f4bb406c3c0 0x7f4bb406c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:16:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.069+0000 7f4bb2ffd640 1 --2- 172.21.15.5:0/1956925389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bb4162600 0x7f4bb4162a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0097a0 tx=0x7f4b9c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef0db09a9b57e381 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.069+0000 7f4bb27fc640 1 -- 172.21.15.5:0/1956925389 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c031830 con 0x7f4bb4162600 2023-11-24T06:16:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.070+0000 7f4bb27fc640 1 -- 172.21.15.5:0/1956925389 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9c031990 con 0x7f4bb4162600 2023-11-24T06:16:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.070+0000 7f4bb27fc640 1 -- 172.21.15.5:0/1956925389 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c031d00 con 0x7f4bb4162600 2023-11-24T06:16:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.070+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1956925389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bb4162600 msgr2=0x7f4bb4162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.070+0000 7f4bba1c9640 1 --2- 172.21.15.5:0/1956925389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bb4162600 0x7f4bb4162a60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0097a0 tx=0x7f4b9c02b8d0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.070+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1956925389 shutdown_connections 2023-11-24T06:16:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.070+0000 7f4bba1c9640 1 --2- 172.21.15.5:0/1956925389 >> v2:172.21.15.6:3300/0 conn(0x7f4bb406c3c0 0x7f4bb406c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.070+0000 7f4bba1c9640 1 --2- 172.21.15.5:0/1956925389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bb4162600 0x7f4bb4162a60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.070+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1956925389 >> 172.21.15.5:0/1956925389 conn(0x7f4bb40b0ba0 msgr2=0x7f4bb40b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.070+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1956925389 shutdown_connections 2023-11-24T06:16:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.070+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1956925389 wait complete. 2023-11-24T06:16:34.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.072+0000 7f4bba1c9640 1 Processor -- start 2023-11-24T06:16:34.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.072+0000 7f4bba1c9640 1 -- start start 2023-11-24T06:16:34.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.073+0000 7f4bba1c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bb406c3c0 0x7f4bb41ebb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.073+0000 7f4bba1c9640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bb4162220 0x7f4bb41531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.073+0000 7f4bba1c9640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bb4162600 0x7f4bb40b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.073+0000 7f4bba1c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb41ec070 con 0x7f4bb406c3c0 2023-11-24T06:16:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.074+0000 7f4bba1c9640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb409ae40 con 0x7f4bb4162220 2023-11-24T06:16:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.074+0000 7f4bba1c9640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb409b120 con 0x7f4bb4162600 2023-11-24T06:16:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.074+0000 7f4bb37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bb406c3c0 0x7f4bb41ebb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.074+0000 7f4bb37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bb406c3c0 0x7f4bb41ebb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50550/0 (socket says 172.21.15.5:50550) 2023-11-24T06:16:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.074+0000 7f4bb37fe640 1 -- 172.21.15.5:0/1193517918 learned_addr learned my addr 172.21.15.5:0/1193517918 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.074+0000 7f4bb3fff640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bb4162600 0x7f4bb40b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.074+0000 7f4bb2ffd640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bb4162220 0x7f4bb41531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.074+0000 7f4bb37fe640 1 -- 172.21.15.5:0/1193517918 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bb4162600 msgr2=0x7f4bb40b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.074+0000 7f4bb37fe640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bb4162600 0x7f4bb40b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.074+0000 7f4bb37fe640 1 -- 172.21.15.5:0/1193517918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bb4162220 msgr2=0x7f4bb41531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.074+0000 7f4bb37fe640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bb4162220 0x7f4bb41531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.074+0000 7f4bb37fe640 1 -- 172.21.15.5:0/1193517918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac009400 con 0x7f4bb406c3c0 2023-11-24T06:16:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.074+0000 7f4bb3fff640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bb4162600 0x7f4bb40b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.075+0000 7f4bb37fe640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bb406c3c0 0x7f4bb41ebb30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4ba400cbc0 tx=0x7f4ba400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.075+0000 7f4babfff640 1 -- 172.21.15.5:0/1193517918 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba400ad40 con 0x7f4bb406c3c0 2023-11-24T06:16:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.075+0000 7f4babfff640 1 -- 172.21.15.5:0/1193517918 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba4012400 con 0x7f4bb406c3c0 2023-11-24T06:16:34.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.075+0000 7f4babfff640 1 -- 172.21.15.5:0/1193517918 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba400aea0 con 0x7f4bb406c3c0 2023-11-24T06:16:34.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.075+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1193517918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9c009400 con 0x7f4bb406c3c0 2023-11-24T06:16:34.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.075+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1193517918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb409b6e0 con 0x7f4bb406c3c0 2023-11-24T06:16:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.076+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1193517918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb409d200 con 0x7f4bb406c3c0 2023-11-24T06:16:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.077+0000 7f4babfff640 1 -- 172.21.15.5:0/1193517918 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4ba4007070 con 0x7f4bb406c3c0 2023-11-24T06:16:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.079+0000 7f4babfff640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4b9406ae40 0x7f4b9406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.079+0000 7f4babfff640 1 -- 172.21.15.5:0/1193517918 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4ba4089700 con 0x7f4bb406c3c0 2023-11-24T06:16:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.079+0000 7f4bb2ffd640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4b9406ae40 0x7f4b9406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:34.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.080+0000 7f4bb2ffd640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4b9406ae40 0x7f4b9406d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c036000 tx=0x7f4b9c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:34.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.083+0000 7f4babfff640 1 -- 172.21.15.5:0/1193517918 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4ba4054710 con 0x7f4bb406c3c0 2023-11-24T06:16:34.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.282+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1193517918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4bb404e920 con 0x7f4bb406c3c0 2023-11-24T06:16:34.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.282+0000 7f4babfff640 1 -- 172.21.15.5:0/1193517918 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4ba4057d30 con 0x7f4bb406c3c0 2023-11-24T06:16:34.285 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:34.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.285+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1193517918 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4b9406ae40 msgr2=0x7f4b9406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:34.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.285+0000 7f4bba1c9640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4b9406ae40 0x7f4b9406d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c036000 tx=0x7f4b9c008030 comp rx=0 tx=0).stop 2023-11-24T06:16:34.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.285+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1193517918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bb406c3c0 msgr2=0x7f4bb41ebb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:34.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.285+0000 7f4bba1c9640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bb406c3c0 0x7f4bb41ebb30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4ba400cbc0 tx=0x7f4ba400a430 comp rx=0 tx=0).stop 2023-11-24T06:16:34.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.285+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1193517918 shutdown_connections 2023-11-24T06:16:34.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.285+0000 7f4bba1c9640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4b9406ae40 0x7f4b9406d300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.285+0000 7f4bba1c9640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4bb4162600 0x7f4bb40b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.285+0000 7f4bba1c9640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4bb4162220 0x7f4bb41531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.285+0000 7f4bba1c9640 1 --2- 172.21.15.5:0/1193517918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bb406c3c0 0x7f4bb41ebb30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.285+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1193517918 >> 172.21.15.5:0/1193517918 conn(0x7f4bb40b0ba0 msgr2=0x7f4bb40aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:34.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.285+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1193517918 shutdown_connections 2023-11-24T06:16:34.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.285+0000 7f4bba1c9640 1 -- 172.21.15.5:0/1193517918 wait complete. 2023-11-24T06:16:34.307 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:34.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.529+0000 7f5182911640 1 Processor -- start 2023-11-24T06:16:34.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.529+0000 7f5182911640 1 -- start start 2023-11-24T06:16:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.531+0000 7f5182911640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f517c0c8930 0x7f517c0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.531+0000 7f5182911640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517c0c6bf0 con 0x7f517c0d1d90 2023-11-24T06:16:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.531+0000 7f5182911640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f517c0cedc0 con 0x7f517c15d3b0 2023-11-24T06:16:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.531+0000 7f5182911640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517c0c6e40 con 0x7f517c0c8930 2023-11-24T06:16:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.532+0000 7f5182110640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f517c0c8930 0x7f517c0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.532+0000 7f518110e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f517c0d1d90 0x7f517c0b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56298/0 (socket says 172.21.15.5:56298) 2023-11-24T06:16:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.532+0000 7f518110e640 1 -- 172.21.15.5:0/1284368831 learned_addr learned my addr 172.21.15.5:0/1284368831 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.532+0000 7f5182110640 1 -- 172.21.15.5:0/1284368831 >> v1:172.21.15.6:6789/0 conn(0x7f517c15d3b0 legacy=0x7f517c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.532+0000 7f5182110640 1 -- 172.21.15.5:0/1284368831 >> v1:172.21.15.5:6789/0 conn(0x7f517c0d1d90 legacy=0x7f517c0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:34.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.532+0000 7f5182110640 1 -- 172.21.15.5:0/1284368831 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c0c7120 con 0x7f517c0c8930 2023-11-24T06:16:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.532+0000 7f518090d640 1 -- 172.21.15.5:0/1284368831 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871169513 0 0) 0x7f517c0c6bf0 con 0x7f517c0d1d90 2023-11-24T06:16:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.533+0000 7f5182110640 1 --2- 172.21.15.5:0/1284368831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f517c0c8930 0x7f517c0ce6c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5174009d90 tx=0x7f517402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b3043468e9740f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.534+0000 7f518090d640 1 -- 172.21.15.5:0/1284368831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517402cbd0 con 0x7f517c0c8930 2023-11-24T06:16:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.534+0000 7f518090d640 1 -- 172.21.15.5:0/1284368831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517402cd30 con 0x7f517c0c8930 2023-11-24T06:16:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.534+0000 7f518090d640 1 -- 172.21.15.5:0/1284368831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517402cea0 con 0x7f517c0c8930 2023-11-24T06:16:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.534+0000 7f5182911640 1 -- 172.21.15.5:0/1284368831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f517c0c8930 msgr2=0x7f517c0ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.534+0000 7f5182911640 1 --2- 172.21.15.5:0/1284368831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f517c0c8930 0x7f517c0ce6c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5174009d90 tx=0x7f517402be20 comp rx=0 tx=0).stop 2023-11-24T06:16:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.535+0000 7f5182911640 1 -- 172.21.15.5:0/1284368831 shutdown_connections 2023-11-24T06:16:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.535+0000 7f5182911640 1 --2- 172.21.15.5:0/1284368831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f517c0c8930 0x7f517c0ce6c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.535+0000 7f5182911640 1 -- 172.21.15.5:0/1284368831 >> 172.21.15.5:0/1284368831 conn(0x7f517c0d30a0 msgr2=0x7f517c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.535+0000 7f5182911640 1 -- 172.21.15.5:0/1284368831 shutdown_connections 2023-11-24T06:16:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.535+0000 7f5182911640 1 -- 172.21.15.5:0/1284368831 wait complete. 2023-11-24T06:16:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.537+0000 7f5182911640 1 Processor -- start 2023-11-24T06:16:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.537+0000 7f5182911640 1 -- start start 2023-11-24T06:16:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.539+0000 7f5182911640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f517c0c8930 0x7f517c0a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.539+0000 7f5182911640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f517c0d1d90 0x7f517c09da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.539+0000 7f5182911640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f517c15d3b0 0x7f517c0b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.539+0000 7f5182911640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0b6fe0 con 0x7f517c0c8930 2023-11-24T06:16:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.539+0000 7f5182911640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0a0980 con 0x7f517c15d3b0 2023-11-24T06:16:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.539+0000 7f5182911640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517c0a0c60 con 0x7f517c0d1d90 2023-11-24T06:16:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.539+0000 7f518190f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f517c0c8930 0x7f517c0a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.539+0000 7f518190f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f517c0c8930 0x7f517c0a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50564/0 (socket says 172.21.15.5:50564) 2023-11-24T06:16:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.539+0000 7f518190f640 1 -- 172.21.15.5:0/1833526985 learned_addr learned my addr 172.21.15.5:0/1833526985 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.539+0000 7f518110e640 1 --2- 172.21.15.5:0/1833526985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f517c0d1d90 0x7f517c09da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.540+0000 7f518190f640 1 -- 172.21.15.5:0/1833526985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f517c0d1d90 msgr2=0x7f517c09da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.540+0000 7f518190f640 1 --2- 172.21.15.5:0/1833526985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f517c0d1d90 0x7f517c09da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.540+0000 7f518190f640 1 -- 172.21.15.5:0/1833526985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f517c15d3b0 msgr2=0x7f517c0b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.540+0000 7f518190f640 1 --2- 172.21.15.5:0/1833526985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f517c15d3b0 0x7f517c0b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.540+0000 7f518190f640 1 -- 172.21.15.5:0/1833526985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5174009a70 con 0x7f517c0c8930 2023-11-24T06:16:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.540+0000 7f518110e640 1 --2- 172.21.15.5:0/1833526985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f517c0d1d90 0x7f517c09da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.540+0000 7f518190f640 1 --2- 172.21.15.5:0/1833526985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f517c0c8930 0x7f517c0a0150 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f517800b3f0 tx=0x7f517800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.540+0000 7f516e7fc640 1 -- 172.21.15.5:0/1833526985 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178011830 con 0x7f517c0c8930 2023-11-24T06:16:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.540+0000 7f516e7fc640 1 -- 172.21.15.5:0/1833526985 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5178011990 con 0x7f517c0c8930 2023-11-24T06:16:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.540+0000 7f5182911640 1 -- 172.21.15.5:0/1833526985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517c0b72a0 con 0x7f517c0c8930 2023-11-24T06:16:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.540+0000 7f516e7fc640 1 -- 172.21.15.5:0/1833526985 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178011d40 con 0x7f517c0c8930 2023-11-24T06:16:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.541+0000 7f5182911640 1 -- 172.21.15.5:0/1833526985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517c09dfa0 con 0x7f517c0c8930 2023-11-24T06:16:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.542+0000 7f5182911640 1 -- 172.21.15.5:0/1833526985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517c09e220 con 0x7f517c0c8930 2023-11-24T06:16:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.543+0000 7f516e7fc640 1 -- 172.21.15.5:0/1833526985 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f517801a400 con 0x7f517c0c8930 2023-11-24T06:16:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.544+0000 7f516e7fc640 1 --2- 172.21.15.5:0/1833526985 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f516006ae40 0x7f516006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.544+0000 7f516e7fc640 1 -- 172.21.15.5:0/1833526985 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f51780894f0 con 0x7f517c0c8930 2023-11-24T06:16:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.544+0000 7f518110e640 1 --2- 172.21.15.5:0/1833526985 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f516006ae40 0x7f516006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.545+0000 7f518110e640 1 --2- 172.21.15.5:0/1833526985 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f516006ae40 0x7f516006d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f517c0d05a0 tx=0x7f5170009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.548+0000 7f516e7fc640 1 -- 172.21.15.5:0/1833526985 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f517c09e220 con 0x7f517c0c8930 2023-11-24T06:16:34.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.743+0000 7f5182911640 1 -- 172.21.15.5:0/1833526985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f517c09e220 con 0x7f517c0c8930 2023-11-24T06:16:34.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.744+0000 7f516e7fc640 1 -- 172.21.15.5:0/1833526985 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f517c09e220 con 0x7f517c0c8930 2023-11-24T06:16:34.748 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.747+0000 7f5182911640 1 -- 172.21.15.5:0/1833526985 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f516006ae40 msgr2=0x7f516006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.747+0000 7f5182911640 1 --2- 172.21.15.5:0/1833526985 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f516006ae40 0x7f516006d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f517c0d05a0 tx=0x7f5170009040 comp rx=0 tx=0).stop 2023-11-24T06:16:34.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.747+0000 7f5182911640 1 -- 172.21.15.5:0/1833526985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f517c0c8930 msgr2=0x7f517c0a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.747+0000 7f5182911640 1 --2- 172.21.15.5:0/1833526985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f517c0c8930 0x7f517c0a0150 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f517800b3f0 tx=0x7f517800b8b0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.747+0000 7f5182911640 1 -- 172.21.15.5:0/1833526985 shutdown_connections 2023-11-24T06:16:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.747+0000 7f5182911640 1 --2- 172.21.15.5:0/1833526985 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f516006ae40 0x7f516006d300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.747+0000 7f5182911640 1 --2- 172.21.15.5:0/1833526985 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f517c0d1d90 0x7f517c09da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.747+0000 7f5182911640 1 --2- 172.21.15.5:0/1833526985 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f517c15d3b0 0x7f517c0b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.747+0000 7f5182911640 1 --2- 172.21.15.5:0/1833526985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f517c0c8930 0x7f517c0a0150 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.747+0000 7f5182911640 1 -- 172.21.15.5:0/1833526985 >> 172.21.15.5:0/1833526985 conn(0x7f517c0d30a0 msgr2=0x7f517c0d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.748+0000 7f5182911640 1 -- 172.21.15.5:0/1833526985 shutdown_connections 2023-11-24T06:16:34.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.748+0000 7f5182911640 1 -- 172.21.15.5:0/1833526985 wait complete. 2023-11-24T06:16:34.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_failed' 2023-11-24T06:16:34.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.989+0000 7fb86169f640 1 Processor -- start 2023-11-24T06:16:34.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.990+0000 7fb86169f640 1 -- start start 2023-11-24T06:16:34.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.991+0000 7fb86169f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb85c15d5b0 0x7fb85c0d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.991+0000 7fb86169f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85c0b1650 0x7fb85c0d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.991+0000 7fb86169f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb85c0cf0c0 con 0x7fb85c0d14a0 2023-11-24T06:16:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.991+0000 7fb86169f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb85c0b01b0 con 0x7fb85c15d5b0 2023-11-24T06:16:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.991+0000 7fb86169f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c15d990 con 0x7fb85c0b1650 2023-11-24T06:16:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.992+0000 7fb85bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb85c15d5b0 0x7fb85c0d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.992+0000 7fb85b7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb85c0d14a0 0x7fb85c0cf7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56300/0 (socket says 172.21.15.5:56300) 2023-11-24T06:16:34.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.992+0000 7fb85b7fe640 1 -- 172.21.15.5:0/3047575561 learned_addr learned my addr 172.21.15.5:0/3047575561 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:34.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.992+0000 7fb860e9e640 1 --2- 172.21.15.5:0/3047575561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85c0b1650 0x7fb85c0d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:34.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.992+0000 7fb85bfff640 1 -- 172.21.15.5:0/3047575561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85c0b1650 msgr2=0x7fb85c0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:34.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.992+0000 7fb85bfff640 1 --2- 172.21.15.5:0/3047575561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85c0b1650 0x7fb85c0d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.992+0000 7fb85bfff640 1 -- 172.21.15.5:0/3047575561 >> v1:172.21.15.5:6789/0 conn(0x7fb85c0d14a0 legacy=0x7fb85c0cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:34.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.992+0000 7fb85bfff640 1 -- 172.21.15.5:0/3047575561 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c0d1e40 con 0x7fb85c15d5b0 2023-11-24T06:16:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.994+0000 7fb85bfff640 1 --2- 172.21.15.5:0/3047575561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb85c15d5b0 0x7fb85c0d3990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009d90 tx=0x7fb84c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7009ad70d68eb0d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.994+0000 7fb85affd640 1 -- 172.21.15.5:0/3047575561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c02cc60 con 0x7fb85c15d5b0 2023-11-24T06:16:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.994+0000 7fb85affd640 1 -- 172.21.15.5:0/3047575561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c02cdc0 con 0x7fb85c15d5b0 2023-11-24T06:16:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.994+0000 7fb85affd640 1 -- 172.21.15.5:0/3047575561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c034660 con 0x7fb85c15d5b0 2023-11-24T06:16:34.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.995+0000 7fb86169f640 1 -- 172.21.15.5:0/3047575561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb85c15d5b0 msgr2=0x7fb85c0d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:34.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.995+0000 7fb86169f640 1 --2- 172.21.15.5:0/3047575561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb85c15d5b0 0x7fb85c0d3990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009d90 tx=0x7fb84c02beb0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.995+0000 7fb86169f640 1 -- 172.21.15.5:0/3047575561 shutdown_connections 2023-11-24T06:16:34.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.995+0000 7fb86169f640 1 --2- 172.21.15.5:0/3047575561 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85c0b1650 0x7fb85c0d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.995+0000 7fb86169f640 1 --2- 172.21.15.5:0/3047575561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb85c15d5b0 0x7fb85c0d3990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:34.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.995+0000 7fb86169f640 1 -- 172.21.15.5:0/3047575561 >> 172.21.15.5:0/3047575561 conn(0x7fb85c09c3b0 msgr2=0x7fb85c0d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:34.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.995+0000 7fb86169f640 1 -- 172.21.15.5:0/3047575561 shutdown_connections 2023-11-24T06:16:34.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.995+0000 7fb86169f640 1 -- 172.21.15.5:0/3047575561 wait complete. 2023-11-24T06:16:35.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.997+0000 7fb86169f640 1 Processor -- start 2023-11-24T06:16:35.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.997+0000 7fb86169f640 1 -- start start 2023-11-24T06:16:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.998+0000 7fb86169f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85c0b1650 0x7fb85c0b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.998+0000 7fb86169f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb85c0d14a0 0x7fb85c0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:35.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.998+0000 7fb86169f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb85c15d5b0 0x7fb85c15bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.998+0000 7fb86169f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0b4380 con 0x7fb85c0d14a0 2023-11-24T06:16:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.998+0000 7fb86169f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0b3700 con 0x7fb85c15d5b0 2023-11-24T06:16:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.998+0000 7fb86169f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0b39e0 con 0x7fb85c0b1650 2023-11-24T06:16:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.999+0000 7fb85b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb85c0d14a0 0x7fb85c0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.999+0000 7fb85b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb85c0d14a0 0x7fb85c0cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50586/0 (socket says 172.21.15.5:50586) 2023-11-24T06:16:35.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.999+0000 7fb85b7fe640 1 -- 172.21.15.5:0/2993327816 learned_addr learned my addr 172.21.15.5:0/2993327816 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:35.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.999+0000 7fb85bfff640 1 --2- 172.21.15.5:0/2993327816 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85c0b1650 0x7fb85c0b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:35.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.999+0000 7fb860e9e640 1 --2- 172.21.15.5:0/2993327816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb85c15d5b0 0x7fb85c15bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:35.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.999+0000 7fb85b7fe640 1 -- 172.21.15.5:0/2993327816 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85c0b1650 msgr2=0x7fb85c0b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.999+0000 7fb85b7fe640 1 --2- 172.21.15.5:0/2993327816 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85c0b1650 0x7fb85c0b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.999+0000 7fb85b7fe640 1 -- 172.21.15.5:0/2993327816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb85c15d5b0 msgr2=0x7fb85c15bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.999+0000 7fb85b7fe640 1 --2- 172.21.15.5:0/2993327816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb85c15d5b0 0x7fb85c15bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.999+0000 7fb85b7fe640 1 -- 172.21.15.5:0/2993327816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c009a70 con 0x7fb85c0d14a0 2023-11-24T06:16:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:34.999+0000 7fb85b7fe640 1 --2- 172.21.15.5:0/2993327816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb85c0d14a0 0x7fb85c0cc770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c4a0 tx=0x7fb84400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.000+0000 7fb858ff9640 1 -- 172.21.15.5:0/2993327816 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844005ce0 con 0x7fb85c0d14a0 2023-11-24T06:16:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.000+0000 7fb858ff9640 1 -- 172.21.15.5:0/2993327816 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb844005e40 con 0x7fb85c0d14a0 2023-11-24T06:16:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.000+0000 7fb858ff9640 1 -- 172.21.15.5:0/2993327816 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb844007270 con 0x7fb85c0d14a0 2023-11-24T06:16:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.000+0000 7fb86169f640 1 -- 172.21.15.5:0/2993327816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85c0b3cc0 con 0x7fb85c0d14a0 2023-11-24T06:16:35.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.000+0000 7fb86169f640 1 -- 172.21.15.5:0/2993327816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85c0cccb0 con 0x7fb85c0d14a0 2023-11-24T06:16:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.001+0000 7fb86169f640 1 -- 172.21.15.5:0/2993327816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85c04e840 con 0x7fb85c0d14a0 2023-11-24T06:16:35.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.002+0000 7fb858ff9640 1 -- 172.21.15.5:0/2993327816 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb844007490 con 0x7fb85c0d14a0 2023-11-24T06:16:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.003+0000 7fb858ff9640 1 --2- 172.21.15.5:0/2993327816 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb83c06ae90 0x7fb83c06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.003+0000 7fb858ff9640 1 -- 172.21.15.5:0/2993327816 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb844088b60 con 0x7fb85c0d14a0 2023-11-24T06:16:35.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.003+0000 7fb85bfff640 1 --2- 172.21.15.5:0/2993327816 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb83c06ae90 0x7fb83c06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:35.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.004+0000 7fb85bfff640 1 --2- 172.21.15.5:0/2993327816 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb83c06ae90 0x7fb83c06d350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb84c036000 tx=0x7fb84c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:35.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.007+0000 7fb858ff9640 1 -- 172.21.15.5:0/2993327816 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb844053ca0 con 0x7fb85c0d14a0 2023-11-24T06:16:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.278+0000 7fb86169f640 1 -- 172.21.15.5:0/2993327816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_failed"]} v 0) v1 -- 0x7fb85c0678d0 con 0x7fb85c0d14a0 2023-11-24T06:16:35.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.981+0000 7fb858ff9640 1 -- 172.21.15.5:0/2993327816 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_failed"]}]=0 v63) v1 ==== 168+0+0 (secure 0 0 0) 0x7fb8440170e0 con 0x7fb85c0d14a0 2023-11-24T06:16:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.984+0000 7fb86169f640 1 -- 172.21.15.5:0/2993327816 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb83c06ae90 msgr2=0x7fb83c06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.984+0000 7fb86169f640 1 --2- 172.21.15.5:0/2993327816 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb83c06ae90 0x7fb83c06d350 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb84c036000 tx=0x7fb84c030000 comp rx=0 tx=0).stop 2023-11-24T06:16:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.984+0000 7fb86169f640 1 -- 172.21.15.5:0/2993327816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb85c0d14a0 msgr2=0x7fb85c0cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.984+0000 7fb86169f640 1 --2- 172.21.15.5:0/2993327816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb85c0d14a0 0x7fb85c0cc770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c4a0 tx=0x7fb84400c960 comp rx=0 tx=0).stop 2023-11-24T06:16:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.984+0000 7fb86169f640 1 -- 172.21.15.5:0/2993327816 shutdown_connections 2023-11-24T06:16:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.984+0000 7fb86169f640 1 --2- 172.21.15.5:0/2993327816 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb83c06ae90 0x7fb83c06d350 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.984+0000 7fb86169f640 1 --2- 172.21.15.5:0/2993327816 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb85c15d5b0 0x7fb85c15bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.984+0000 7fb86169f640 1 --2- 172.21.15.5:0/2993327816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb85c0d14a0 0x7fb85c0cc770 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.984+0000 7fb86169f640 1 --2- 172.21.15.5:0/2993327816 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb85c0b1650 0x7fb85c0b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.984+0000 7fb86169f640 1 -- 172.21.15.5:0/2993327816 >> 172.21.15.5:0/2993327816 conn(0x7fb85c09c3b0 msgr2=0x7fb85c0cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:35.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.985+0000 7fb86169f640 1 -- 172.21.15.5:0/2993327816 shutdown_connections 2023-11-24T06:16:35.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:35.985+0000 7fb86169f640 1 -- 172.21.15.5:0/2993327816 wait complete. 2023-11-24T06:16:36.008 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:16:36.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.239+0000 7f3a133f4640 1 Processor -- start 2023-11-24T06:16:36.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.239+0000 7f3a133f4640 1 -- start start 2023-11-24T06:16:36.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.240+0000 7f3a133f4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a0c15d340 0x7f3a0c0d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:36.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.240+0000 7f3a133f4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a0c0cdbb0 0x7f3a0c0d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:36.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.240+0000 7f3a133f4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3a0c0d1bc0 0x7f3a0c0b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:36.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.240+0000 7f3a133f4640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0c0d2050 con 0x7f3a0c0cdbb0 2023-11-24T06:16:36.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.240+0000 7f3a133f4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0c0cef30 con 0x7f3a0c0d1bc0 2023-11-24T06:16:36.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.240+0000 7f3a133f4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c0d11a0 con 0x7f3a0c15d340 2023-11-24T06:16:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.241+0000 7f3a11bf1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a0c0cdbb0 0x7f3a0c0d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.241+0000 7f3a11bf1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a0c0cdbb0 0x7f3a0c0d1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50600/0 (socket says 172.21.15.5:50600) 2023-11-24T06:16:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.241+0000 7f3a11bf1640 1 -- 172.21.15.5:0/1509845359 learned_addr learned my addr 172.21.15.5:0/1509845359 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.241+0000 7f3a123f2640 1 --2- 172.21.15.5:0/1509845359 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a0c15d340 0x7f3a0c0d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.241+0000 7f3a12bf3640 1 --2- 172.21.15.5:0/1509845359 >> v2:172.21.15.6:3300/0 conn(0x7f3a0c0d1bc0 0x7f3a0c0b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.241+0000 7f3a11bf1640 1 -- 172.21.15.5:0/1509845359 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a0c15d340 msgr2=0x7f3a0c0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.241+0000 7f3a11bf1640 1 --2- 172.21.15.5:0/1509845359 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a0c15d340 0x7f3a0c0d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.241+0000 7f3a11bf1640 1 -- 172.21.15.5:0/1509845359 >> v2:172.21.15.6:3300/0 conn(0x7f3a0c0d1bc0 msgr2=0x7f3a0c0b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.241+0000 7f3a11bf1640 1 --2- 172.21.15.5:0/1509845359 >> v2:172.21.15.6:3300/0 conn(0x7f3a0c0d1bc0 0x7f3a0c0b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.241+0000 7f3a11bf1640 1 -- 172.21.15.5:0/1509845359 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c0ce320 con 0x7f3a0c0cdbb0 2023-11-24T06:16:36.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.241+0000 7f3a12bf3640 1 --2- 172.21.15.5:0/1509845359 >> v2:172.21.15.6:3300/0 conn(0x7f3a0c0d1bc0 0x7f3a0c0b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:36.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.243+0000 7f3a11bf1640 1 --2- 172.21.15.5:0/1509845359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0c0cdbb0 0x7f3a0c0d1680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0b9120 tx=0x7f39fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f38f668ecb4e89 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.243+0000 7f3a113f0640 1 -- 172.21.15.5:0/1509845359 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc031870 con 0x7f3a0c0cdbb0 2023-11-24T06:16:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.243+0000 7f3a113f0640 1 -- 172.21.15.5:0/1509845359 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc0319d0 con 0x7f3a0c0cdbb0 2023-11-24T06:16:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.243+0000 7f3a113f0640 1 -- 172.21.15.5:0/1509845359 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc031d60 con 0x7f3a0c0cdbb0 2023-11-24T06:16:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.244+0000 7f3a133f4640 1 -- 172.21.15.5:0/1509845359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0c0cdbb0 msgr2=0x7f3a0c0d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.244+0000 7f3a133f4640 1 --2- 172.21.15.5:0/1509845359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0c0cdbb0 0x7f3a0c0d1680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0b9120 tx=0x7f39fc02b910 comp rx=0 tx=0).stop 2023-11-24T06:16:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.244+0000 7f3a133f4640 1 -- 172.21.15.5:0/1509845359 shutdown_connections 2023-11-24T06:16:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.244+0000 7f3a133f4640 1 --2- 172.21.15.5:0/1509845359 >> v2:172.21.15.6:3300/0 conn(0x7f3a0c0d1bc0 0x7f3a0c0b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.244+0000 7f3a133f4640 1 --2- 172.21.15.5:0/1509845359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0c0cdbb0 0x7f3a0c0d1680 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.244+0000 7f3a133f4640 1 --2- 172.21.15.5:0/1509845359 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a0c15d340 0x7f3a0c0d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.244+0000 7f3a133f4640 1 -- 172.21.15.5:0/1509845359 >> 172.21.15.5:0/1509845359 conn(0x7f3a0c09c370 msgr2=0x7f3a0c0d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.244+0000 7f3a133f4640 1 -- 172.21.15.5:0/1509845359 shutdown_connections 2023-11-24T06:16:36.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.244+0000 7f3a133f4640 1 -- 172.21.15.5:0/1509845359 wait complete. 2023-11-24T06:16:36.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.246+0000 7f3a133f4640 1 Processor -- start 2023-11-24T06:16:36.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.246+0000 7f3a133f4640 1 -- start start 2023-11-24T06:16:36.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.248+0000 7f3a133f4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a0c0cdbb0 0x7f3a0c159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:36.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.248+0000 7f3a133f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0c0d1bc0 0x7f3a0c0b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:36.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.248+0000 7f3a133f4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a0c15d340 0x7f3a0c0b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:36.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.248+0000 7f3a133f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c1f3560 con 0x7f3a0c0d1bc0 2023-11-24T06:16:36.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.248+0000 7f3a133f4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c15a470 con 0x7f3a0c15d340 2023-11-24T06:16:36.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.248+0000 7f3a133f4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c0b54e0 con 0x7f3a0c0cdbb0 2023-11-24T06:16:36.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.248+0000 7f3a11bf1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0c0d1bc0 0x7f3a0c0b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:36.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.248+0000 7f3a11bf1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0c0d1bc0 0x7f3a0c0b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50616/0 (socket says 172.21.15.5:50616) 2023-11-24T06:16:36.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.248+0000 7f3a11bf1640 1 -- 172.21.15.5:0/801862025 learned_addr learned my addr 172.21.15.5:0/801862025 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:36.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.248+0000 7f3a123f2640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a0c0cdbb0 0x7f3a0c159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:36.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.248+0000 7f3a12bf3640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a0c15d340 0x7f3a0c0b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:36.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.248+0000 7f3a11bf1640 1 -- 172.21.15.5:0/801862025 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a0c0cdbb0 msgr2=0x7f3a0c159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:36.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.248+0000 7f3a11bf1640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a0c0cdbb0 0x7f3a0c159f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:36.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.248+0000 7f3a11bf1640 1 -- 172.21.15.5:0/801862025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a0c15d340 msgr2=0x7f3a0c0b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:36.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.249+0000 7f3a11bf1640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a0c15d340 0x7f3a0c0b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:36.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.249+0000 7f3a11bf1640 1 -- 172.21.15.5:0/801862025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc0094d0 con 0x7f3a0c0d1bc0 2023-11-24T06:16:36.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.249+0000 7f3a12bf3640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a0c15d340 0x7f3a0c0b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:36.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.249+0000 7f3a11bf1640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0c0d1bc0 0x7f3a0c0b0de0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f39fc02b950 tx=0x7f39fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:36.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.249+0000 7f3a02ffd640 1 -- 172.21.15.5:0/801862025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc03ad30 con 0x7f3a0c0d1bc0 2023-11-24T06:16:36.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.249+0000 7f3a02ffd640 1 -- 172.21.15.5:0/801862025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc0443f0 con 0x7f3a0c0d1bc0 2023-11-24T06:16:36.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.249+0000 7f3a02ffd640 1 -- 172.21.15.5:0/801862025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc0447a0 con 0x7f3a0c0d1bc0 2023-11-24T06:16:36.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.249+0000 7f3a133f4640 1 -- 172.21.15.5:0/801862025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0c0b4690 con 0x7f3a0c0d1bc0 2023-11-24T06:16:36.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.249+0000 7f3a133f4640 1 -- 172.21.15.5:0/801862025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0c0b4bb0 con 0x7f3a0c0d1bc0 2023-11-24T06:16:36.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.251+0000 7f3a133f4640 1 -- 172.21.15.5:0/801862025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0c067d70 con 0x7f3a0c0d1bc0 2023-11-24T06:16:36.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.252+0000 7f3a02ffd640 1 -- 172.21.15.5:0/801862025 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f39fc04d450 con 0x7f3a0c0d1bc0 2023-11-24T06:16:36.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.252+0000 7f3a02ffd640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f39f006ae40 0x7f39f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:36.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.252+0000 7f3a02ffd640 1 -- 172.21.15.5:0/801862025 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f39fc02d040 con 0x7f3a0c0d1bc0 2023-11-24T06:16:36.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.253+0000 7f3a123f2640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f39f006ae40 0x7f39f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:36.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.253+0000 7f3a123f2640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f39f006ae40 0x7f39f006d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3a08006fd0 tx=0x7f3a08008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:36.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.257+0000 7f3a02ffd640 1 -- 172.21.15.5:0/801862025 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f39fc076ae0 con 0x7f3a0c0d1bc0 2023-11-24T06:16:36.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.460+0000 7f3a133f4640 1 -- 172.21.15.5:0/801862025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3a0c0b4690 con 0x7f3a0c0d1bc0 2023-11-24T06:16:36.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.466+0000 7f3a02ffd640 1 -- 172.21.15.5:0/801862025 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3a0c0b4690 con 0x7f3a0c0d1bc0 2023-11-24T06:16:36.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.468+0000 7f3a133f4640 1 -- 172.21.15.5:0/801862025 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f39f006ae40 msgr2=0x7f39f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.468+0000 7f3a133f4640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f39f006ae40 0x7f39f006d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3a08006fd0 tx=0x7f3a08008040 comp rx=0 tx=0).stop 2023-11-24T06:16:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.469+0000 7f3a133f4640 1 -- 172.21.15.5:0/801862025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0c0d1bc0 msgr2=0x7f3a0c0b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.469+0000 7f3a133f4640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0c0d1bc0 0x7f3a0c0b0de0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f39fc02b950 tx=0x7f39fc0046c0 comp rx=0 tx=0).stop 2023-11-24T06:16:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.469+0000 7f3a133f4640 1 -- 172.21.15.5:0/801862025 shutdown_connections 2023-11-24T06:16:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.469+0000 7f3a133f4640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f39f006ae40 0x7f39f006d300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.469+0000 7f3a133f4640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a0c15d340 0x7f3a0c0b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.469+0000 7f3a133f4640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a0c0d1bc0 0x7f3a0c0b0de0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.469+0000 7f3a133f4640 1 --2- 172.21.15.5:0/801862025 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a0c0cdbb0 0x7f3a0c159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:36.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.469+0000 7f3a133f4640 1 -- 172.21.15.5:0/801862025 >> 172.21.15.5:0/801862025 conn(0x7f3a0c09c370 msgr2=0x7f3a0c15bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.469+0000 7f3a133f4640 1 -- 172.21.15.5:0/801862025 shutdown_connections 2023-11-24T06:16:36.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:36.469+0000 7f3a133f4640 1 -- 172.21.15.5:0/801862025 wait complete. 2023-11-24T06:16:36.492 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:16:36.492 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:16:36.601 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:16:36.740 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:16:36.741 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:16:36.741 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:16:36.807 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:16:36.807 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:16:36.863 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:37.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.074+0000 7f3d8359e640 1 Processor -- start 2023-11-24T06:16:37.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.074+0000 7f3d8359e640 1 -- start start 2023-11-24T06:16:37.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.076+0000 7f3d8359e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d8414b1c0 0x7f3d84162670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:37.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.076+0000 7f3d8359e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3d840cf760 0x7f3d841655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:37.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.076+0000 7f3d8359e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d84162bb0 con 0x7f3d841651a0 2023-11-24T06:16:37.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.076+0000 7f3d8359e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d84165ca0 con 0x7f3d840cf760 2023-11-24T06:16:37.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.076+0000 7f3d8359e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d84165f80 con 0x7f3d8414b1c0 2023-11-24T06:16:37.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.076+0000 7f3d81d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3d841651a0 0x7f3d840cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56302/0 (socket says 172.21.15.5:56302) 2023-11-24T06:16:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.076+0000 7f3d81d9b640 1 -- 172.21.15.5:0/2536660037 learned_addr learned my addr 172.21.15.5:0/2536660037 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.076+0000 7f3d82d9d640 1 --2- 172.21.15.5:0/2536660037 >> v2:172.21.15.5:3300/0 conn(0x7f3d840cf760 0x7f3d841655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:37.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.077+0000 7f3d8259c640 1 --2- 172.21.15.5:0/2536660037 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d8414b1c0 0x7f3d84162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.077+0000 7f3d8159a640 1 -- 172.21.15.5:0/2536660037 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221316500 0 0) 0x7f3d84162bb0 con 0x7f3d841651a0 2023-11-24T06:16:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.077+0000 7f3d8159a640 1 -- 172.21.15.5:0/2536660037 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d6c003520 con 0x7f3d841651a0 2023-11-24T06:16:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.077+0000 7f3d82d9d640 1 -- 172.21.15.5:0/2536660037 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d8414b1c0 msgr2=0x7f3d84162670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.077+0000 7f3d82d9d640 1 --2- 172.21.15.5:0/2536660037 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d8414b1c0 0x7f3d84162670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.077+0000 7f3d82d9d640 1 -- 172.21.15.5:0/2536660037 >> v1:172.21.15.5:6789/0 conn(0x7f3d841651a0 legacy=0x7f3d840cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.077+0000 7f3d82d9d640 1 -- 172.21.15.5:0/2536660037 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d84161e70 con 0x7f3d840cf760 2023-11-24T06:16:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.078+0000 7f3d82d9d640 1 --2- 172.21.15.5:0/2536660037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d840cf760 0x7f3d841655e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0097a0 tx=0x7f3d7c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8192d66198282d23 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.078+0000 7f3d8159a640 1 -- 172.21.15.5:0/2536660037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c031780 con 0x7f3d840cf760 2023-11-24T06:16:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.078+0000 7f3d8159a640 1 -- 172.21.15.5:0/2536660037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7c0318e0 con 0x7f3d840cf760 2023-11-24T06:16:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.078+0000 7f3d8159a640 1 -- 172.21.15.5:0/2536660037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c031c70 con 0x7f3d840cf760 2023-11-24T06:16:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.079+0000 7f3d8359e640 1 -- 172.21.15.5:0/2536660037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d840cf760 msgr2=0x7f3d841655e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.079+0000 7f3d8359e640 1 --2- 172.21.15.5:0/2536660037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d840cf760 0x7f3d841655e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c0097a0 tx=0x7f3d7c02b820 comp rx=0 tx=0).stop 2023-11-24T06:16:37.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.079+0000 7f3d8359e640 1 -- 172.21.15.5:0/2536660037 shutdown_connections 2023-11-24T06:16:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.079+0000 7f3d8359e640 1 --2- 172.21.15.5:0/2536660037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d840cf760 0x7f3d841655e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.079+0000 7f3d8359e640 1 --2- 172.21.15.5:0/2536660037 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d8414b1c0 0x7f3d84162670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.079+0000 7f3d8359e640 1 -- 172.21.15.5:0/2536660037 >> 172.21.15.5:0/2536660037 conn(0x7f3d8406a860 msgr2=0x7f3d8406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.079+0000 7f3d8359e640 1 -- 172.21.15.5:0/2536660037 shutdown_connections 2023-11-24T06:16:37.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.079+0000 7f3d8359e640 1 -- 172.21.15.5:0/2536660037 wait complete. 2023-11-24T06:16:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.081+0000 7f3d8359e640 1 Processor -- start 2023-11-24T06:16:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.081+0000 7f3d8359e640 1 -- start start 2023-11-24T06:16:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.082+0000 7f3d8359e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d840cf760 0x7f3d840b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.082+0000 7f3d8359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d8414b1c0 0x7f3d8415afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.082+0000 7f3d8359e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d841651a0 0x7f3d8415e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:37.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.082+0000 7f3d8359e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d841fb530 con 0x7f3d8414b1c0 2023-11-24T06:16:37.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.082+0000 7f3d8359e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8415de80 con 0x7f3d841651a0 2023-11-24T06:16:37.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.082+0000 7f3d8359e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d840b1630 con 0x7f3d840cf760 2023-11-24T06:16:37.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.083+0000 7f3d81d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d8414b1c0 0x7f3d8415afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:37.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.083+0000 7f3d81d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d8414b1c0 0x7f3d8415afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50634/0 (socket says 172.21.15.5:50634) 2023-11-24T06:16:37.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.083+0000 7f3d81d9b640 1 -- 172.21.15.5:0/2625449253 learned_addr learned my addr 172.21.15.5:0/2625449253 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.083+0000 7f3d8259c640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d840cf760 0x7f3d840b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.083+0000 7f3d82d9d640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d841651a0 0x7f3d8415e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.083+0000 7f3d81d9b640 1 -- 172.21.15.5:0/2625449253 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d840cf760 msgr2=0x7f3d840b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.083+0000 7f3d81d9b640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d840cf760 0x7f3d840b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.083+0000 7f3d81d9b640 1 -- 172.21.15.5:0/2625449253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d841651a0 msgr2=0x7f3d8415e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.083+0000 7f3d81d9b640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d841651a0 0x7f3d8415e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.083+0000 7f3d81d9b640 1 -- 172.21.15.5:0/2625449253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c009400 con 0x7f3d8414b1c0 2023-11-24T06:16:37.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.083+0000 7f3d82d9d640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d841651a0 0x7f3d8415e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:37.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.083+0000 7f3d81d9b640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d8414b1c0 0x7f3d8415afa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800c440 tx=0x7f3d7800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:37.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.084+0000 7f3d72ffd640 1 -- 172.21.15.5:0/2625449253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78005ce0 con 0x7f3d8414b1c0 2023-11-24T06:16:37.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.084+0000 7f3d72ffd640 1 -- 172.21.15.5:0/2625449253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d78005e40 con 0x7f3d8414b1c0 2023-11-24T06:16:37.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.084+0000 7f3d72ffd640 1 -- 172.21.15.5:0/2625449253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d780072b0 con 0x7f3d8414b1c0 2023-11-24T06:16:37.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.084+0000 7f3d8359e640 1 -- 172.21.15.5:0/2625449253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8415bc50 con 0x7f3d8414b1c0 2023-11-24T06:16:37.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.084+0000 7f3d8359e640 1 -- 172.21.15.5:0/2625449253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d840ce140 con 0x7f3d8414b1c0 2023-11-24T06:16:37.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.085+0000 7f3d8359e640 1 -- 172.21.15.5:0/2625449253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8406c5c0 con 0x7f3d8414b1c0 2023-11-24T06:16:37.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.086+0000 7f3d72ffd640 1 -- 172.21.15.5:0/2625449253 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3d78007410 con 0x7f3d8414b1c0 2023-11-24T06:16:37.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.087+0000 7f3d72ffd640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3d6406ae40 0x7f3d6406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:37.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.087+0000 7f3d72ffd640 1 -- 172.21.15.5:0/2625449253 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3d78088c40 con 0x7f3d8414b1c0 2023-11-24T06:16:37.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.087+0000 7f3d8259c640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3d6406ae40 0x7f3d6406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:37.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.088+0000 7f3d8259c640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3d6406ae40 0x7f3d6406d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009a60 tx=0x7f3d74003990 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:37.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.092+0000 7f3d72ffd640 1 -- 172.21.15.5:0/2625449253 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d78053c80 con 0x7f3d8414b1c0 2023-11-24T06:16:37.257 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:16:37.267 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:16:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.293+0000 7f3d8359e640 1 -- 172.21.15.5:0/2625449253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d840b1d30 con 0x7f3d8414b1c0 2023-11-24T06:16:37.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.295+0000 7f3d72ffd640 1 -- 172.21.15.5:0/2625449253 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16658 (secure 0 0 0) 0x7f3d78017020 con 0x7f3d8414b1c0 2023-11-24T06:16:37.298 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:37.298 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:16:33.958286+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:37.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.297+0000 7f3d8359e640 1 -- 172.21.15.5:0/2625449253 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3d6406ae40 msgr2=0x7f3d6406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:37.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.297+0000 7f3d8359e640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3d6406ae40 0x7f3d6406d300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009a60 tx=0x7f3d74003990 comp rx=0 tx=0).stop 2023-11-24T06:16:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.297+0000 7f3d8359e640 1 -- 172.21.15.5:0/2625449253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d8414b1c0 msgr2=0x7f3d8415afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.297+0000 7f3d8359e640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d8414b1c0 0x7f3d8415afa0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800c440 tx=0x7f3d7800c900 comp rx=0 tx=0).stop 2023-11-24T06:16:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.297+0000 7f3d8359e640 1 -- 172.21.15.5:0/2625449253 shutdown_connections 2023-11-24T06:16:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.297+0000 7f3d8359e640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3d6406ae40 0x7f3d6406d300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.297+0000 7f3d8359e640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3d841651a0 0x7f3d8415e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.297+0000 7f3d8359e640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d8414b1c0 0x7f3d8415afa0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.297+0000 7f3d8359e640 1 --2- 172.21.15.5:0/2625449253 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3d840cf760 0x7f3d840b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.297+0000 7f3d8359e640 1 -- 172.21.15.5:0/2625449253 >> 172.21.15.5:0/2625449253 conn(0x7f3d8406a860 msgr2=0x7f3d84071bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.298+0000 7f3d8359e640 1 -- 172.21.15.5:0/2625449253 shutdown_connections 2023-11-24T06:16:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.298+0000 7f3d8359e640 1 -- 172.21.15.5:0/2625449253 wait complete. 2023-11-24T06:16:37.321 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:37.321 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.577+0000 7f9932e1e640 1 Processor -- start 2023-11-24T06:16:37.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.577+0000 7f9932e1e640 1 -- start start 2023-11-24T06:16:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.579+0000 7f9932e1e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f992c06cad0 0x7f992c06e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.579+0000 7f9932e1e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f992c072280 0x7f992c0b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.579+0000 7f9932e1e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992c071780 con 0x7f992c160b10 2023-11-24T06:16:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.579+0000 7f9932e1e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992c0b2320 con 0x7f992c06cad0 2023-11-24T06:16:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.579+0000 7f9932e1e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992c06df30 con 0x7f992c072280 2023-11-24T06:16:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.580+0000 7f992bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f992c06cad0 0x7f992c06e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.580+0000 7f992bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f992c06cad0 0x7f992c06e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50650/0 (socket says 172.21.15.5:50650) 2023-11-24T06:16:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.580+0000 7f992bfff640 1 -- 172.21.15.5:0/418862453 learned_addr learned my addr 172.21.15.5:0/418862453 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.580+0000 7f99313bb640 1 --2- 172.21.15.5:0/418862453 >> v2:172.21.15.6:3300/0 conn(0x7f992c072280 0x7f992c0b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.580+0000 7f992bfff640 1 -- 172.21.15.5:0/418862453 >> v2:172.21.15.6:3300/0 conn(0x7f992c072280 msgr2=0x7f992c0b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.580+0000 7f992bfff640 1 --2- 172.21.15.5:0/418862453 >> v2:172.21.15.6:3300/0 conn(0x7f992c072280 0x7f992c0b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.580+0000 7f992bfff640 1 -- 172.21.15.5:0/418862453 >> v1:172.21.15.5:6789/0 conn(0x7f992c160b10 legacy=0x7f992c06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:37.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.580+0000 7f992bfff640 1 -- 172.21.15.5:0/418862453 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c0711c0 con 0x7f992c06cad0 2023-11-24T06:16:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.581+0000 7f992bfff640 1 --2- 172.21.15.5:0/418862453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f992c06cad0 0x7f992c06e420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f992c0b3fe0 tx=0x7f991802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7799b2fc3906b56a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.582+0000 7f992b7fe640 1 -- 172.21.15.5:0/418862453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99180317e0 con 0x7f992c06cad0 2023-11-24T06:16:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.582+0000 7f992b7fe640 1 -- 172.21.15.5:0/418862453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9918031940 con 0x7f992c06cad0 2023-11-24T06:16:37.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.582+0000 7f992b7fe640 1 -- 172.21.15.5:0/418862453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9918031cd0 con 0x7f992c06cad0 2023-11-24T06:16:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.583+0000 7f9932e1e640 1 -- 172.21.15.5:0/418862453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f992c06cad0 msgr2=0x7f992c06e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.583+0000 7f9932e1e640 1 --2- 172.21.15.5:0/418862453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f992c06cad0 0x7f992c06e420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f992c0b3fe0 tx=0x7f991802b880 comp rx=0 tx=0).stop 2023-11-24T06:16:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.583+0000 7f9932e1e640 1 -- 172.21.15.5:0/418862453 shutdown_connections 2023-11-24T06:16:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.583+0000 7f9932e1e640 1 --2- 172.21.15.5:0/418862453 >> v2:172.21.15.6:3300/0 conn(0x7f992c072280 0x7f992c0b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.583+0000 7f9932e1e640 1 --2- 172.21.15.5:0/418862453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f992c06cad0 0x7f992c06e420 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.583+0000 7f9932e1e640 1 -- 172.21.15.5:0/418862453 >> 172.21.15.5:0/418862453 conn(0x7f992c0b0ba0 msgr2=0x7f992c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.583+0000 7f9932e1e640 1 -- 172.21.15.5:0/418862453 shutdown_connections 2023-11-24T06:16:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.583+0000 7f9932e1e640 1 -- 172.21.15.5:0/418862453 wait complete. 2023-11-24T06:16:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.585+0000 7f9932e1e640 1 Processor -- start 2023-11-24T06:16:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.586+0000 7f9932e1e640 1 -- start start 2023-11-24T06:16:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.587+0000 7f9932e1e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f992c06cad0 0x7f992c1f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.587+0000 7f9932e1e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f992c072280 0x7f992c0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.587+0000 7f9932e1e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f992c160b10 0x7f992c099a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.587+0000 7f9932e1e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c151410 con 0x7f992c160b10 2023-11-24T06:16:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.587+0000 7f9932e1e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c0ce940 con 0x7f992c06cad0 2023-11-24T06:16:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.587+0000 7f9932e1e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992c1f2c20 con 0x7f992c072280 2023-11-24T06:16:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.587+0000 7f99313bb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f992c160b10 0x7f992c099a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.587+0000 7f99313bb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f992c160b10 0x7f992c099a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50666/0 (socket says 172.21.15.5:50666) 2023-11-24T06:16:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.588+0000 7f99313bb640 1 -- 172.21.15.5:0/377940921 learned_addr learned my addr 172.21.15.5:0/377940921 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.588+0000 7f9930bba640 1 --2- 172.21.15.5:0/377940921 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f992c06cad0 0x7f992c1f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.588+0000 7f99313bb640 1 -- 172.21.15.5:0/377940921 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f992c072280 msgr2=0x7f992c0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.588+0000 7f99313bb640 1 --2- 172.21.15.5:0/377940921 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f992c072280 0x7f992c0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.588+0000 7f99313bb640 1 -- 172.21.15.5:0/377940921 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f992c06cad0 msgr2=0x7f992c1f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.588+0000 7f99313bb640 1 --2- 172.21.15.5:0/377940921 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f992c06cad0 0x7f992c1f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.588+0000 7f99313bb640 1 -- 172.21.15.5:0/377940921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99180094d0 con 0x7f992c160b10 2023-11-24T06:16:37.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.588+0000 7f99313bb640 1 --2- 172.21.15.5:0/377940921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f992c160b10 0x7f992c099a00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9924007e40 tx=0x7f992400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:37.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.589+0000 7f99297fa640 1 -- 172.21.15.5:0/377940921 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924014620 con 0x7f992c160b10 2023-11-24T06:16:37.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.589+0000 7f99297fa640 1 -- 172.21.15.5:0/377940921 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99240147a0 con 0x7f992c160b10 2023-11-24T06:16:37.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.589+0000 7f99297fa640 1 -- 172.21.15.5:0/377940921 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924014b50 con 0x7f992c160b10 2023-11-24T06:16:37.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.589+0000 7f9932e1e640 1 -- 172.21.15.5:0/377940921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992c158b90 con 0x7f992c160b10 2023-11-24T06:16:37.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.589+0000 7f9932e1e640 1 -- 172.21.15.5:0/377940921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992c0ce660 con 0x7f992c160b10 2023-11-24T06:16:37.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.590+0000 7f9932e1e640 1 -- 172.21.15.5:0/377940921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992c09d200 con 0x7f992c160b10 2023-11-24T06:16:37.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.591+0000 7f99297fa640 1 -- 172.21.15.5:0/377940921 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9924014cb0 con 0x7f992c160b10 2023-11-24T06:16:37.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.592+0000 7f99297fa640 1 --2- 172.21.15.5:0/377940921 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f991006ae90 0x7f991006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:37.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.592+0000 7f99297fa640 1 -- 172.21.15.5:0/377940921 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9924089fa0 con 0x7f992c160b10 2023-11-24T06:16:37.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.593+0000 7f9930bba640 1 --2- 172.21.15.5:0/377940921 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f991006ae90 0x7f991006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:37.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.593+0000 7f9930bba640 1 --2- 172.21.15.5:0/377940921 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f991006ae90 0x7f991006d350 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9920006fd0 tx=0x7f9920008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:37.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.597+0000 7f99297fa640 1 -- 172.21.15.5:0/377940921 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9924055030 con 0x7f992c160b10 2023-11-24T06:16:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.792+0000 7f9932e1e640 1 -- 172.21.15.5:0/377940921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f992c04e920 con 0x7f992c160b10 2023-11-24T06:16:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.793+0000 7f99297fa640 1 -- 172.21.15.5:0/377940921 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9924058650 con 0x7f992c160b10 2023-11-24T06:16:37.796 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.795+0000 7f9932e1e640 1 -- 172.21.15.5:0/377940921 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f991006ae90 msgr2=0x7f991006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.795+0000 7f9932e1e640 1 --2- 172.21.15.5:0/377940921 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f991006ae90 0x7f991006d350 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9920006fd0 tx=0x7f9920008040 comp rx=0 tx=0).stop 2023-11-24T06:16:37.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.795+0000 7f9932e1e640 1 -- 172.21.15.5:0/377940921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f992c160b10 msgr2=0x7f992c099a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.795+0000 7f9932e1e640 1 --2- 172.21.15.5:0/377940921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f992c160b10 0x7f992c099a00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9924007e40 tx=0x7f992400e4b0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.795+0000 7f9932e1e640 1 -- 172.21.15.5:0/377940921 shutdown_connections 2023-11-24T06:16:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.795+0000 7f9932e1e640 1 --2- 172.21.15.5:0/377940921 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f991006ae90 0x7f991006d350 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.795+0000 7f9932e1e640 1 --2- 172.21.15.5:0/377940921 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f992c072280 0x7f992c0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.795+0000 7f9932e1e640 1 --2- 172.21.15.5:0/377940921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f992c160b10 0x7f992c099a00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.795+0000 7f9932e1e640 1 --2- 172.21.15.5:0/377940921 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f992c06cad0 0x7f992c1f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.795+0000 7f9932e1e640 1 -- 172.21.15.5:0/377940921 >> 172.21.15.5:0/377940921 conn(0x7f992c0b0ba0 msgr2=0x7f992c06c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.796+0000 7f9932e1e640 1 -- 172.21.15.5:0/377940921 shutdown_connections 2023-11-24T06:16:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:37.796+0000 7f9932e1e640 1 -- 172.21.15.5:0/377940921 wait complete. 2023-11-24T06:16:37.818 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:38.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.042+0000 7f85fe9ea640 1 Processor -- start 2023-11-24T06:16:38.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.043+0000 7f85fe9ea640 1 -- start start 2023-11-24T06:16:38.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.044+0000 7f85fe9ea640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f85f8146bd0 0x7f85f80ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.044+0000 7f85fe9ea640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85f8161b80 0x7f85f815e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.045+0000 7f85fe9ea640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f85f806e700 0x7f85f806eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.045+0000 7f85fe9ea640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f806f0b0 con 0x7f85f806e700 2023-11-24T06:16:38.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.045+0000 7f85fe9ea640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f815de10 con 0x7f85f8146bd0 2023-11-24T06:16:38.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.045+0000 7f85fe9ea640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f80c5f90 con 0x7f85f8161b80 2023-11-24T06:16:38.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.045+0000 7f85fcf87640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f85f806e700 0x7f85f806eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.045+0000 7f85fcf87640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f85f806e700 0x7f85f806eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50676/0 (socket says 172.21.15.5:50676) 2023-11-24T06:16:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.045+0000 7f85fcf87640 1 -- 172.21.15.5:0/1983864665 learned_addr learned my addr 172.21.15.5:0/1983864665 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.045+0000 7f85f7fff640 1 --2- 172.21.15.5:0/1983864665 >> v2:172.21.15.6:3300/0 conn(0x7f85f8146bd0 0x7f85f80ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.045+0000 7f85f77fe640 1 --2- 172.21.15.5:0/1983864665 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85f8161b80 0x7f85f815e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.045+0000 7f85fcf87640 1 -- 172.21.15.5:0/1983864665 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85f8161b80 msgr2=0x7f85f815e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.045+0000 7f85fcf87640 1 --2- 172.21.15.5:0/1983864665 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85f8161b80 0x7f85f815e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.045+0000 7f85fcf87640 1 -- 172.21.15.5:0/1983864665 >> v2:172.21.15.6:3300/0 conn(0x7f85f8146bd0 msgr2=0x7f85f80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.045+0000 7f85fcf87640 1 --2- 172.21.15.5:0/1983864665 >> v2:172.21.15.6:3300/0 conn(0x7f85f8146bd0 0x7f85f80ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.045+0000 7f85fcf87640 1 -- 172.21.15.5:0/1983864665 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f80c6270 con 0x7f85f806e700 2023-11-24T06:16:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.047+0000 7f85fcf87640 1 --2- 172.21.15.5:0/1983864665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85f806e700 0x7f85f806eb70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f85f8066da0 tx=0x7f85f002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ebe62ce0fa4cab server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.047+0000 7f85f6ffd640 1 -- 172.21.15.5:0/1983864665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0031860 con 0x7f85f806e700 2023-11-24T06:16:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.047+0000 7f85f6ffd640 1 -- 172.21.15.5:0/1983864665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f00319c0 con 0x7f85f806e700 2023-11-24T06:16:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.047+0000 7f85f6ffd640 1 -- 172.21.15.5:0/1983864665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0031d50 con 0x7f85f806e700 2023-11-24T06:16:38.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.048+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1983864665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85f806e700 msgr2=0x7f85f806eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.048+0000 7f85fe9ea640 1 --2- 172.21.15.5:0/1983864665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85f806e700 0x7f85f806eb70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f85f8066da0 tx=0x7f85f002b900 comp rx=0 tx=0).stop 2023-11-24T06:16:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.048+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1983864665 shutdown_connections 2023-11-24T06:16:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.048+0000 7f85fe9ea640 1 --2- 172.21.15.5:0/1983864665 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85f8161b80 0x7f85f815e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.048+0000 7f85fe9ea640 1 --2- 172.21.15.5:0/1983864665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85f806e700 0x7f85f806eb70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.048+0000 7f85fe9ea640 1 --2- 172.21.15.5:0/1983864665 >> v2:172.21.15.6:3300/0 conn(0x7f85f8146bd0 0x7f85f80ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.048+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1983864665 >> 172.21.15.5:0/1983864665 conn(0x7f85f80661f0 msgr2=0x7f85f80665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.048+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1983864665 shutdown_connections 2023-11-24T06:16:38.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.048+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1983864665 wait complete. 2023-11-24T06:16:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.050+0000 7f85fe9ea640 1 Processor -- start 2023-11-24T06:16:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.050+0000 7f85fe9ea640 1 -- start start 2023-11-24T06:16:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.051+0000 7f85fe9ea640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85f806e700 0x7f85f80d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.051+0000 7f85fe9ea640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f8146bd0 0x7f85f81e7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.051+0000 7f85fe9ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85f8161b80 0x7f85f80ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.051+0000 7f85fe9ea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f81f3e60 con 0x7f85f8161b80 2023-11-24T06:16:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.051+0000 7f85fe9ea640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f81f3250 con 0x7f85f8146bd0 2023-11-24T06:16:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.051+0000 7f85fe9ea640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f81f3530 con 0x7f85f806e700 2023-11-24T06:16:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.051+0000 7f85fcf87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85f8161b80 0x7f85f80ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.051+0000 7f85fcf87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85f8161b80 0x7f85f80ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50680/0 (socket says 172.21.15.5:50680) 2023-11-24T06:16:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.051+0000 7f85fcf87640 1 -- 172.21.15.5:0/1836173091 learned_addr learned my addr 172.21.15.5:0/1836173091 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:38.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.052+0000 7f85f7fff640 1 --2- 172.21.15.5:0/1836173091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85f806e700 0x7f85f80d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.052+0000 7f85f77fe640 1 --2- 172.21.15.5:0/1836173091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f8146bd0 0x7f85f81e7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.052+0000 7f85fcf87640 1 -- 172.21.15.5:0/1836173091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85f806e700 msgr2=0x7f85f80d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.052+0000 7f85fcf87640 1 --2- 172.21.15.5:0/1836173091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85f806e700 0x7f85f80d01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.052+0000 7f85fcf87640 1 -- 172.21.15.5:0/1836173091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f8146bd0 msgr2=0x7f85f81e7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.052+0000 7f85fcf87640 1 --2- 172.21.15.5:0/1836173091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f8146bd0 0x7f85f81e7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.052+0000 7f85fcf87640 1 -- 172.21.15.5:0/1836173091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f00094d0 con 0x7f85f8161b80 2023-11-24T06:16:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.052+0000 7f85fcf87640 1 --2- 172.21.15.5:0/1836173091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85f8161b80 0x7f85f80ced20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f85f00090b0 tx=0x7f85f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.053+0000 7f85f4ff9640 1 -- 172.21.15.5:0/1836173091 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f003ac60 con 0x7f85f8161b80 2023-11-24T06:16:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.053+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1836173091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f81f3810 con 0x7f85f8161b80 2023-11-24T06:16:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.053+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1836173091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f80d06e0 con 0x7f85f8161b80 2023-11-24T06:16:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.053+0000 7f85f4ff9640 1 -- 172.21.15.5:0/1836173091 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f003adc0 con 0x7f85f8161b80 2023-11-24T06:16:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.053+0000 7f85f4ff9640 1 -- 172.21.15.5:0/1836173091 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f00445f0 con 0x7f85f8161b80 2023-11-24T06:16:38.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.054+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1836173091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85dc005250 con 0x7f85f8161b80 2023-11-24T06:16:38.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.055+0000 7f85f4ff9640 1 -- 172.21.15.5:0/1836173091 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85f0044750 con 0x7f85f8161b80 2023-11-24T06:16:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.056+0000 7f85f4ff9640 1 --2- 172.21.15.5:0/1836173091 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f85d806ae40 0x7f85d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.056+0000 7f85f4ff9640 1 -- 172.21.15.5:0/1836173091 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f85f00abce0 con 0x7f85f8161b80 2023-11-24T06:16:38.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.057+0000 7f85f7fff640 1 --2- 172.21.15.5:0/1836173091 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f85d806ae40 0x7f85d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.057+0000 7f85f7fff640 1 --2- 172.21.15.5:0/1836173091 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f85d806ae40 0x7f85d806d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f85f80b0880 tx=0x7f85e8003a10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:38.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.061+0000 7f85f4ff9640 1 -- 172.21.15.5:0/1836173091 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85f0076c60 con 0x7f85f8161b80 2023-11-24T06:16:38.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.254+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1836173091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85dc005a80 con 0x7f85f8161b80 2023-11-24T06:16:38.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.255+0000 7f85f4ff9640 1 -- 172.21.15.5:0/1836173091 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f85f0035020 con 0x7f85f8161b80 2023-11-24T06:16:38.258 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:38.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.258+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1836173091 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f85d806ae40 msgr2=0x7f85d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:38.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.258+0000 7f85fe9ea640 1 --2- 172.21.15.5:0/1836173091 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f85d806ae40 0x7f85d806d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f85f80b0880 tx=0x7f85e8003a10 comp rx=0 tx=0).stop 2023-11-24T06:16:38.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.258+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1836173091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85f8161b80 msgr2=0x7f85f80ced20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:38.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.258+0000 7f85fe9ea640 1 --2- 172.21.15.5:0/1836173091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85f8161b80 0x7f85f80ced20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f85f00090b0 tx=0x7f85f00046c0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.258+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1836173091 shutdown_connections 2023-11-24T06:16:38.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.258+0000 7f85fe9ea640 1 --2- 172.21.15.5:0/1836173091 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f85d806ae40 0x7f85d806d300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.259+0000 7f85fe9ea640 1 --2- 172.21.15.5:0/1836173091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85f8161b80 0x7f85f80ced20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.259+0000 7f85fe9ea640 1 --2- 172.21.15.5:0/1836173091 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85f8146bd0 0x7f85f81e7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.259+0000 7f85fe9ea640 1 --2- 172.21.15.5:0/1836173091 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85f806e700 0x7f85f80d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.259+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1836173091 >> 172.21.15.5:0/1836173091 conn(0x7f85f80661f0 msgr2=0x7f85f80665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:38.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.259+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1836173091 shutdown_connections 2023-11-24T06:16:38.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.259+0000 7f85fe9ea640 1 -- 172.21.15.5:0/1836173091 wait complete. 2023-11-24T06:16:38.279 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:38.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.501+0000 7f6c57e94640 1 Processor -- start 2023-11-24T06:16:38.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.501+0000 7f6c57e94640 1 -- start start 2023-11-24T06:16:38.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.502+0000 7f6c57e94640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c50160b10 0x7f6c5006e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.503+0000 7f6c57e94640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f6c50072280 0x7f6c5006c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.503+0000 7f6c57e94640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c5006c900 0x7f6c50160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.503+0000 7f6c57e94640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c500b2320 con 0x7f6c5006c900 2023-11-24T06:16:38.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.503+0000 7f6c57e94640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5006cd90 con 0x7f6c50072280 2023-11-24T06:16:38.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.503+0000 7f6c57e94640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5006df30 con 0x7f6c50160b10 2023-11-24T06:16:38.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.503+0000 7f6c56431640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c5006c900 0x7f6c50160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.503+0000 7f6c56431640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6c5006c900 0x7f6c50160fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50692/0 (socket says 172.21.15.5:50692) 2023-11-24T06:16:38.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.503+0000 7f6c56431640 1 -- 172.21.15.5:0/578410334 learned_addr learned my addr 172.21.15.5:0/578410334 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:38.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.503+0000 7f6c5542f640 1 --2- 172.21.15.5:0/578410334 >> v2:172.21.15.6:3300/0 conn(0x7f6c50072280 0x7f6c5006c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.503+0000 7f6c55c30640 1 --2- 172.21.15.5:0/578410334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c50160b10 0x7f6c5006e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.504+0000 7f6c56431640 1 -- 172.21.15.5:0/578410334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c50160b10 msgr2=0x7f6c5006e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:38.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.504+0000 7f6c56431640 1 --2- 172.21.15.5:0/578410334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c50160b10 0x7f6c5006e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.504+0000 7f6c56431640 1 -- 172.21.15.5:0/578410334 >> v2:172.21.15.6:3300/0 conn(0x7f6c50072280 msgr2=0x7f6c5006c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:38.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.504+0000 7f6c56431640 1 --2- 172.21.15.5:0/578410334 >> v2:172.21.15.6:3300/0 conn(0x7f6c50072280 0x7f6c5006c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.504+0000 7f6c56431640 1 -- 172.21.15.5:0/578410334 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c50071780 con 0x7f6c5006c900 2023-11-24T06:16:38.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.505+0000 7f6c56431640 1 --2- 172.21.15.5:0/578410334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c5006c900 0x7f6c50160fb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6c500b5130 tx=0x7f6c4802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6bc160e71b37da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:38.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.505+0000 7f6c54c2e640 1 -- 172.21.15.5:0/578410334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4802c9e0 con 0x7f6c5006c900 2023-11-24T06:16:38.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.505+0000 7f6c54c2e640 1 -- 172.21.15.5:0/578410334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4802cb40 con 0x7f6c5006c900 2023-11-24T06:16:38.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.505+0000 7f6c54c2e640 1 -- 172.21.15.5:0/578410334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480343f0 con 0x7f6c5006c900 2023-11-24T06:16:38.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.506+0000 7f6c57e94640 1 -- 172.21.15.5:0/578410334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c5006c900 msgr2=0x7f6c50160fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:38.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.506+0000 7f6c57e94640 1 --2- 172.21.15.5:0/578410334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c5006c900 0x7f6c50160fb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6c500b5130 tx=0x7f6c4802be40 comp rx=0 tx=0).stop 2023-11-24T06:16:38.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.506+0000 7f6c57e94640 1 -- 172.21.15.5:0/578410334 shutdown_connections 2023-11-24T06:16:38.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.506+0000 7f6c57e94640 1 --2- 172.21.15.5:0/578410334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c5006c900 0x7f6c50160fb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.506+0000 7f6c57e94640 1 --2- 172.21.15.5:0/578410334 >> v2:172.21.15.6:3300/0 conn(0x7f6c50072280 0x7f6c5006c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.506+0000 7f6c57e94640 1 --2- 172.21.15.5:0/578410334 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c50160b10 0x7f6c5006e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.506+0000 7f6c57e94640 1 -- 172.21.15.5:0/578410334 >> 172.21.15.5:0/578410334 conn(0x7f6c500b0ba0 msgr2=0x7f6c500b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:38.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.506+0000 7f6c57e94640 1 -- 172.21.15.5:0/578410334 shutdown_connections 2023-11-24T06:16:38.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.506+0000 7f6c57e94640 1 -- 172.21.15.5:0/578410334 wait complete. 2023-11-24T06:16:38.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.508+0000 7f6c57e94640 1 Processor -- start 2023-11-24T06:16:38.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.508+0000 7f6c57e94640 1 -- start start 2023-11-24T06:16:38.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.509+0000 7f6c57e94640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c5006c900 0x7f6c501f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.509+0000 7f6c57e94640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c50072280 0x7f6c500ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.509+0000 7f6c57e94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c50160b10 0x7f6c5009b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.509+0000 7f6c57e94640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c501f3ea0 con 0x7f6c50160b10 2023-11-24T06:16:38.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.509+0000 7f6c57e94640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c501f3290 con 0x7f6c50072280 2023-11-24T06:16:38.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.509+0000 7f6c57e94640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c501f3570 con 0x7f6c5006c900 2023-11-24T06:16:38.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.509+0000 7f6c56431640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c50160b10 0x7f6c5009b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.509+0000 7f6c56431640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c50160b10 0x7f6c5009b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50704/0 (socket says 172.21.15.5:50704) 2023-11-24T06:16:38.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.509+0000 7f6c56431640 1 -- 172.21.15.5:0/583677771 learned_addr learned my addr 172.21.15.5:0/583677771 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:38.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.509+0000 7f6c55c30640 1 --2- 172.21.15.5:0/583677771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c5006c900 0x7f6c501f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.509+0000 7f6c5542f640 1 --2- 172.21.15.5:0/583677771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c50072280 0x7f6c500ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.510+0000 7f6c56431640 1 -- 172.21.15.5:0/583677771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c5006c900 msgr2=0x7f6c501f6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:38.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.510+0000 7f6c56431640 1 --2- 172.21.15.5:0/583677771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c5006c900 0x7f6c501f6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.510+0000 7f6c56431640 1 -- 172.21.15.5:0/583677771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c50072280 msgr2=0x7f6c500ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:38.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.510+0000 7f6c56431640 1 --2- 172.21.15.5:0/583677771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c50072280 0x7f6c500ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.510+0000 7f6c56431640 1 -- 172.21.15.5:0/583677771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c48009b30 con 0x7f6c50160b10 2023-11-24T06:16:38.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.510+0000 7f6c56431640 1 --2- 172.21.15.5:0/583677771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c50160b10 0x7f6c5009b910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6c48036040 tx=0x7f6c48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:38.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.510+0000 7f6c467fc640 1 -- 172.21.15.5:0/583677771 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48009510 con 0x7f6c50160b10 2023-11-24T06:16:38.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.510+0000 7f6c467fc640 1 -- 172.21.15.5:0/583677771 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4803e450 con 0x7f6c50160b10 2023-11-24T06:16:38.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.510+0000 7f6c467fc640 1 -- 172.21.15.5:0/583677771 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4803e7e0 con 0x7f6c50160b10 2023-11-24T06:16:38.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.510+0000 7f6c57e94640 1 -- 172.21.15.5:0/583677771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c501f3850 con 0x7f6c50160b10 2023-11-24T06:16:38.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.511+0000 7f6c57e94640 1 -- 172.21.15.5:0/583677771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c501f2f40 con 0x7f6c50160b10 2023-11-24T06:16:38.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.512+0000 7f6c57e94640 1 -- 172.21.15.5:0/583677771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5009d200 con 0x7f6c50160b10 2023-11-24T06:16:38.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.513+0000 7f6c467fc640 1 -- 172.21.15.5:0/583677771 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c4803e940 con 0x7f6c50160b10 2023-11-24T06:16:38.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.513+0000 7f6c467fc640 1 --2- 172.21.15.5:0/583677771 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6c3406ae40 0x7f6c3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.514+0000 7f6c467fc640 1 -- 172.21.15.5:0/583677771 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6c480ac520 con 0x7f6c50160b10 2023-11-24T06:16:38.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.514+0000 7f6c55c30640 1 --2- 172.21.15.5:0/583677771 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6c3406ae40 0x7f6c3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.515+0000 7f6c55c30640 1 --2- 172.21.15.5:0/583677771 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6c3406ae40 0x7f6c3406d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c006fd0 tx=0x7f6c4c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:38.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.518+0000 7f6c467fc640 1 -- 172.21.15.5:0/583677771 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6c480774a0 con 0x7f6c50160b10 2023-11-24T06:16:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.711+0000 7f6c57e94640 1 -- 172.21.15.5:0/583677771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6c5004e920 con 0x7f6c50160b10 2023-11-24T06:16:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.712+0000 7f6c467fc640 1 -- 172.21.15.5:0/583677771 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6c4807aac0 con 0x7f6c50160b10 2023-11-24T06:16:38.715 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.714+0000 7f6c57e94640 1 -- 172.21.15.5:0/583677771 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6c3406ae40 msgr2=0x7f6c3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.714+0000 7f6c57e94640 1 --2- 172.21.15.5:0/583677771 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6c3406ae40 0x7f6c3406d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c006fd0 tx=0x7f6c4c008040 comp rx=0 tx=0).stop 2023-11-24T06:16:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.714+0000 7f6c57e94640 1 -- 172.21.15.5:0/583677771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c50160b10 msgr2=0x7f6c5009b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.714+0000 7f6c57e94640 1 --2- 172.21.15.5:0/583677771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c50160b10 0x7f6c5009b910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6c48036040 tx=0x7f6c48004a40 comp rx=0 tx=0).stop 2023-11-24T06:16:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.715+0000 7f6c57e94640 1 -- 172.21.15.5:0/583677771 shutdown_connections 2023-11-24T06:16:38.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.715+0000 7f6c57e94640 1 --2- 172.21.15.5:0/583677771 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6c3406ae40 0x7f6c3406d300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.715+0000 7f6c57e94640 1 --2- 172.21.15.5:0/583677771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c50160b10 0x7f6c5009b910 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.715+0000 7f6c57e94640 1 --2- 172.21.15.5:0/583677771 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6c50072280 0x7f6c500ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.715+0000 7f6c57e94640 1 --2- 172.21.15.5:0/583677771 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6c5006c900 0x7f6c501f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.715+0000 7f6c57e94640 1 -- 172.21.15.5:0/583677771 >> 172.21.15.5:0/583677771 conn(0x7f6c500b0ba0 msgr2=0x7f6c500acc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:38.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.715+0000 7f6c57e94640 1 -- 172.21.15.5:0/583677771 shutdown_connections 2023-11-24T06:16:38.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.715+0000 7f6c57e94640 1 -- 172.21.15.5:0/583677771 wait complete. 2023-11-24T06:16:38.738 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:16:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.962+0000 7f0c1a988640 1 Processor -- start 2023-11-24T06:16:38.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.963+0000 7f0c1a988640 1 -- start start 2023-11-24T06:16:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.964+0000 7f0c1a988640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0c14070550 0x7f0c14160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.964+0000 7f0c1a988640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0c14161530 0x7f0c140ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.964+0000 7f0c1a988640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c14161940 con 0x7f0c140b2550 2023-11-24T06:16:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.964+0000 7f0c1a988640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c140b3040 con 0x7f0c14070550 2023-11-24T06:16:38.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.964+0000 7f0c1a988640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c14070cf0 con 0x7f0c14161530 2023-11-24T06:16:38.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.964+0000 7f0c19185640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0c14070550 0x7f0c14160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.964+0000 7f0c19986640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0c140b2550 0x7f0c140b2930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56320/0 (socket says 172.21.15.5:56320) 2023-11-24T06:16:38.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.964+0000 7f0c19986640 1 -- 172.21.15.5:0/2769278347 learned_addr learned my addr 172.21.15.5:0/2769278347 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:38.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.965+0000 7f0c1a187640 1 --2- 172.21.15.5:0/2769278347 >> v2:172.21.15.6:3300/0 conn(0x7f0c14161530 0x7f0c140ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.965+0000 7f0c19185640 1 -- 172.21.15.5:0/2769278347 >> v2:172.21.15.6:3300/0 conn(0x7f0c14161530 msgr2=0x7f0c140ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:38.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.965+0000 7f0c19185640 1 --2- 172.21.15.5:0/2769278347 >> v2:172.21.15.6:3300/0 conn(0x7f0c14161530 0x7f0c140ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.965+0000 7f0c19185640 1 -- 172.21.15.5:0/2769278347 >> v1:172.21.15.5:6789/0 conn(0x7f0c140b2550 legacy=0x7f0c140b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:38.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.965+0000 7f0c19185640 1 -- 172.21.15.5:0/2769278347 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c14070fd0 con 0x7f0c14070550 2023-11-24T06:16:38.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.965+0000 7f0c18984640 1 -- 172.21.15.5:0/2769278347 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675006668 0 0) 0x7f0c14161940 con 0x7f0c140b2550 2023-11-24T06:16:38.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.965+0000 7f0c1a187640 1 --2- 172.21.15.5:0/2769278347 >> v2:172.21.15.6:3300/0 conn(0x7f0c14161530 0x7f0c140ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:38.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.966+0000 7f0c19185640 1 --2- 172.21.15.5:0/2769278347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c14070550 0x7f0c14160ff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0c1415b5d0 tx=0x7f0c0402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f94e7a43396403a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:38.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.966+0000 7f0c18984640 1 -- 172.21.15.5:0/2769278347 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04031830 con 0x7f0c14070550 2023-11-24T06:16:38.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.966+0000 7f0c18984640 1 -- 172.21.15.5:0/2769278347 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c04031990 con 0x7f0c14070550 2023-11-24T06:16:38.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.966+0000 7f0c18984640 1 -- 172.21.15.5:0/2769278347 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04031d20 con 0x7f0c14070550 2023-11-24T06:16:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.967+0000 7f0c1a988640 1 -- 172.21.15.5:0/2769278347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c14070550 msgr2=0x7f0c14160ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.967+0000 7f0c1a988640 1 --2- 172.21.15.5:0/2769278347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c14070550 0x7f0c14160ff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0c1415b5d0 tx=0x7f0c0402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.967+0000 7f0c1a988640 1 -- 172.21.15.5:0/2769278347 shutdown_connections 2023-11-24T06:16:38.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.967+0000 7f0c1a988640 1 --2- 172.21.15.5:0/2769278347 >> v2:172.21.15.6:3300/0 conn(0x7f0c14161530 0x7f0c140ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.967+0000 7f0c1a988640 1 --2- 172.21.15.5:0/2769278347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c14070550 0x7f0c14160ff0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.967+0000 7f0c1a988640 1 -- 172.21.15.5:0/2769278347 >> 172.21.15.5:0/2769278347 conn(0x7f0c140b0840 msgr2=0x7f0c140b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:38.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.967+0000 7f0c1a988640 1 -- 172.21.15.5:0/2769278347 shutdown_connections 2023-11-24T06:16:38.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.967+0000 7f0c1a988640 1 -- 172.21.15.5:0/2769278347 wait complete. 2023-11-24T06:16:38.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.969+0000 7f0c1a988640 1 Processor -- start 2023-11-24T06:16:38.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.969+0000 7f0c1a988640 1 -- start start 2023-11-24T06:16:38.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.970+0000 7f0c1a988640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c14070550 0x7f0c1415f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.970+0000 7f0c1a988640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c140b2550 0x7f0c1415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.970+0000 7f0c1a988640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c14161530 0x7f0c141510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.970+0000 7f0c1a988640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1415a720 con 0x7f0c14070550 2023-11-24T06:16:38.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.970+0000 7f0c1a988640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c14159140 con 0x7f0c140b2550 2023-11-24T06:16:38.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.970+0000 7f0c1a988640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c14159420 con 0x7f0c14161530 2023-11-24T06:16:38.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.970+0000 7f0c19986640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c14070550 0x7f0c1415f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.970+0000 7f0c19986640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c14070550 0x7f0c1415f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50722/0 (socket says 172.21.15.5:50722) 2023-11-24T06:16:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.970+0000 7f0c19986640 1 -- 172.21.15.5:0/2933136389 learned_addr learned my addr 172.21.15.5:0/2933136389 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.970+0000 7f0c1a187640 1 --2- 172.21.15.5:0/2933136389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c14161530 0x7f0c141510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.971+0000 7f0c19185640 1 --2- 172.21.15.5:0/2933136389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c140b2550 0x7f0c1415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.971+0000 7f0c19986640 1 -- 172.21.15.5:0/2933136389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c14161530 msgr2=0x7f0c141510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.971+0000 7f0c19986640 1 --2- 172.21.15.5:0/2933136389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c14161530 0x7f0c141510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.971+0000 7f0c19986640 1 -- 172.21.15.5:0/2933136389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c140b2550 msgr2=0x7f0c1415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.971+0000 7f0c19986640 1 --2- 172.21.15.5:0/2933136389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c140b2550 0x7f0c1415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.971+0000 7f0c19986640 1 -- 172.21.15.5:0/2933136389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c04009400 con 0x7f0c14070550 2023-11-24T06:16:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.971+0000 7f0c19986640 1 --2- 172.21.15.5:0/2933136389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c14070550 0x7f0c1415f6d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000c4d0 tx=0x7f0c1000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.971+0000 7f0c0a7fc640 1 -- 172.21.15.5:0/2933136389 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100074e0 con 0x7f0c14070550 2023-11-24T06:16:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.971+0000 7f0c0a7fc640 1 -- 172.21.15.5:0/2933136389 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c10007660 con 0x7f0c14070550 2023-11-24T06:16:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.971+0000 7f0c0a7fc640 1 -- 172.21.15.5:0/2933136389 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10004a20 con 0x7f0c14070550 2023-11-24T06:16:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.971+0000 7f0c1a988640 1 -- 172.21.15.5:0/2933136389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c14159700 con 0x7f0c14070550 2023-11-24T06:16:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.972+0000 7f0c1a988640 1 -- 172.21.15.5:0/2933136389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c14159cc0 con 0x7f0c14070550 2023-11-24T06:16:38.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.973+0000 7f0c1a988640 1 -- 172.21.15.5:0/2933136389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1404e840 con 0x7f0c14070550 2023-11-24T06:16:38.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.974+0000 7f0c0a7fc640 1 -- 172.21.15.5:0/2933136389 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0c10004b80 con 0x7f0c14070550 2023-11-24T06:16:38.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.975+0000 7f0c0a7fc640 1 --2- 172.21.15.5:0/2933136389 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0bf806ae40 0x7f0bf806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:38.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.975+0000 7f0c0a7fc640 1 -- 172.21.15.5:0/2933136389 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0c10088800 con 0x7f0c14070550 2023-11-24T06:16:38.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.975+0000 7f0c19185640 1 --2- 172.21.15.5:0/2933136389 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0bf806ae40 0x7f0bf806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:38.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.976+0000 7f0c19185640 1 --2- 172.21.15.5:0/2933136389 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0bf806ae40 0x7f0bf806d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0c04036000 tx=0x7f0c0403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:38.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:38.980+0000 7f0c0a7fc640 1 -- 172.21.15.5:0/2933136389 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0c100538b0 con 0x7f0c14070550 2023-11-24T06:16:39.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.193+0000 7f0c1a988640 1 -- 172.21.15.5:0/2933136389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0c140b2b40 con 0x7f0c14070550 2023-11-24T06:16:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.194+0000 7f0c0a7fc640 1 -- 172.21.15.5:0/2933136389 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0c10018020 con 0x7f0c14070550 2023-11-24T06:16:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2023-11-24T06:16:39.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.197+0000 7f0c1a988640 1 -- 172.21.15.5:0/2933136389 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0bf806ae40 msgr2=0x7f0bf806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:39.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.197+0000 7f0c1a988640 1 --2- 172.21.15.5:0/2933136389 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0bf806ae40 0x7f0bf806d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0c04036000 tx=0x7f0c0403c040 comp rx=0 tx=0).stop 2023-11-24T06:16:39.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.197+0000 7f0c1a988640 1 -- 172.21.15.5:0/2933136389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c14070550 msgr2=0x7f0c1415f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:39.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.197+0000 7f0c1a988640 1 --2- 172.21.15.5:0/2933136389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c14070550 0x7f0c1415f6d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000c4d0 tx=0x7f0c1000c990 comp rx=0 tx=0).stop 2023-11-24T06:16:39.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.197+0000 7f0c1a988640 1 -- 172.21.15.5:0/2933136389 shutdown_connections 2023-11-24T06:16:39.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.197+0000 7f0c1a988640 1 --2- 172.21.15.5:0/2933136389 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0bf806ae40 0x7f0bf806d300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.197+0000 7f0c1a988640 1 --2- 172.21.15.5:0/2933136389 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0c14161530 0x7f0c141510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.197+0000 7f0c1a988640 1 --2- 172.21.15.5:0/2933136389 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0c140b2550 0x7f0c1415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.197+0000 7f0c1a988640 1 --2- 172.21.15.5:0/2933136389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c14070550 0x7f0c1415f6d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.197+0000 7f0c1a988640 1 -- 172.21.15.5:0/2933136389 >> 172.21.15.5:0/2933136389 conn(0x7f0c140b0840 msgr2=0x7f0c1415c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:39.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.197+0000 7f0c1a988640 1 -- 172.21.15.5:0/2933136389 shutdown_connections 2023-11-24T06:16:39.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.197+0000 7f0c1a988640 1 -- 172.21.15.5:0/2933136389 wait complete. 2023-11-24T06:16:39.220 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:16:39.220 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:16:39.273 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:39.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.496+0000 7f38f9bbc640 1 Processor -- start 2023-11-24T06:16:39.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.496+0000 7f38f9bbc640 1 -- start start 2023-11-24T06:16:39.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.497+0000 7f38f9bbc640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f38f415d5b0 0x7f38f40d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:39.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.497+0000 7f38f9bbc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f38f40d14a0 0x7f38f40d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:39.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.497+0000 7f38f9bbc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38f40cf7a0 0x7f38f40cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:39.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.497+0000 7f38f9bbc640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f40cf0c0 con 0x7f38f40d14a0 2023-11-24T06:16:39.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.497+0000 7f38f9bbc640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f40d1e40 con 0x7f38f415d5b0 2023-11-24T06:16:39.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.497+0000 7f38f9bbc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40b01b0 con 0x7f38f40cf7a0 2023-11-24T06:16:39.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.497+0000 7f38f3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f38f40d14a0 0x7f38f40d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:39.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.497+0000 7f38f3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f38f40d14a0 0x7f38f40d1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50732/0 (socket says 172.21.15.5:50732) 2023-11-24T06:16:39.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.497+0000 7f38f3fff640 1 -- 172.21.15.5:0/1216890009 learned_addr learned my addr 172.21.15.5:0/1216890009 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.498+0000 7f38f93bb640 1 --2- 172.21.15.5:0/1216890009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38f40cf7a0 0x7f38f40cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.498+0000 7f38f8bba640 1 --2- 172.21.15.5:0/1216890009 >> v2:172.21.15.6:3300/0 conn(0x7f38f415d5b0 0x7f38f40d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.498+0000 7f38f3fff640 1 -- 172.21.15.5:0/1216890009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38f40cf7a0 msgr2=0x7f38f40cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.498+0000 7f38f3fff640 1 --2- 172.21.15.5:0/1216890009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38f40cf7a0 0x7f38f40cfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.498+0000 7f38f3fff640 1 -- 172.21.15.5:0/1216890009 >> v2:172.21.15.6:3300/0 conn(0x7f38f415d5b0 msgr2=0x7f38f40d3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.498+0000 7f38f3fff640 1 --2- 172.21.15.5:0/1216890009 >> v2:172.21.15.6:3300/0 conn(0x7f38f415d5b0 0x7f38f40d3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.498+0000 7f38f3fff640 1 -- 172.21.15.5:0/1216890009 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f415d990 con 0x7f38f40d14a0 2023-11-24T06:16:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.498+0000 7f38f8bba640 1 --2- 172.21.15.5:0/1216890009 >> v2:172.21.15.6:3300/0 conn(0x7f38f415d5b0 0x7f38f40d3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.498+0000 7f38f93bb640 1 --2- 172.21.15.5:0/1216890009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38f40cf7a0 0x7f38f40cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:39.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.499+0000 7f38f3fff640 1 --2- 172.21.15.5:0/1216890009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38f40d14a0 0x7f38f40d1900 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f38f408c9e0 tx=0x7f38e802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a3dc2c5d0adcbc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:39.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.500+0000 7f38f37fe640 1 -- 172.21.15.5:0/1216890009 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8031640 con 0x7f38f40d14a0 2023-11-24T06:16:39.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.500+0000 7f38f37fe640 1 -- 172.21.15.5:0/1216890009 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e80317a0 con 0x7f38f40d14a0 2023-11-24T06:16:39.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.500+0000 7f38f37fe640 1 -- 172.21.15.5:0/1216890009 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8031b30 con 0x7f38f40d14a0 2023-11-24T06:16:39.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.500+0000 7f38f9bbc640 1 -- 172.21.15.5:0/1216890009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38f40d14a0 msgr2=0x7f38f40d1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:39.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.500+0000 7f38f9bbc640 1 --2- 172.21.15.5:0/1216890009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38f40d14a0 0x7f38f40d1900 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f38f408c9e0 tx=0x7f38e802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.500+0000 7f38f9bbc640 1 -- 172.21.15.5:0/1216890009 shutdown_connections 2023-11-24T06:16:39.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.500+0000 7f38f9bbc640 1 --2- 172.21.15.5:0/1216890009 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38f40cf7a0 0x7f38f40cfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.500+0000 7f38f9bbc640 1 --2- 172.21.15.5:0/1216890009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38f40d14a0 0x7f38f40d1900 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.500+0000 7f38f9bbc640 1 --2- 172.21.15.5:0/1216890009 >> v2:172.21.15.6:3300/0 conn(0x7f38f415d5b0 0x7f38f40d3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.500+0000 7f38f9bbc640 1 -- 172.21.15.5:0/1216890009 >> 172.21.15.5:0/1216890009 conn(0x7f38f409c3b0 msgr2=0x7f38f40d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:39.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.500+0000 7f38f9bbc640 1 -- 172.21.15.5:0/1216890009 shutdown_connections 2023-11-24T06:16:39.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.500+0000 7f38f9bbc640 1 -- 172.21.15.5:0/1216890009 wait complete. 2023-11-24T06:16:39.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.502+0000 7f38f9bbc640 1 Processor -- start 2023-11-24T06:16:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.502+0000 7f38f9bbc640 1 -- start start 2023-11-24T06:16:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.503+0000 7f38f9bbc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38f40cf7a0 0x7f38f40c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.503+0000 7f38f9bbc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38f40d14a0 0x7f38f41f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.503+0000 7f38f9bbc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38f415d5b0 0x7f38f406b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.503+0000 7f38f9bbc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40c92f0 con 0x7f38f415d5b0 2023-11-24T06:16:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.503+0000 7f38f9bbc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40c8670 con 0x7f38f40cf7a0 2023-11-24T06:16:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.503+0000 7f38f9bbc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f40c8950 con 0x7f38f40d14a0 2023-11-24T06:16:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.504+0000 7f38f93bb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38f415d5b0 0x7f38f406b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.504+0000 7f38f93bb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38f415d5b0 0x7f38f406b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50736/0 (socket says 172.21.15.5:50736) 2023-11-24T06:16:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.504+0000 7f38f93bb640 1 -- 172.21.15.5:0/127873893 learned_addr learned my addr 172.21.15.5:0/127873893 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.504+0000 7f38f8bba640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38f40cf7a0 0x7f38f40c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.504+0000 7f38f3fff640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38f40d14a0 0x7f38f41f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.504+0000 7f38f93bb640 1 -- 172.21.15.5:0/127873893 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38f40d14a0 msgr2=0x7f38f41f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.504+0000 7f38f93bb640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38f40d14a0 0x7f38f41f3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.504+0000 7f38f93bb640 1 -- 172.21.15.5:0/127873893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38f40cf7a0 msgr2=0x7f38f40c7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.504+0000 7f38f93bb640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38f40cf7a0 0x7f38f40c7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.504+0000 7f38f93bb640 1 -- 172.21.15.5:0/127873893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e80094d0 con 0x7f38f415d5b0 2023-11-24T06:16:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.504+0000 7f38f8bba640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38f40cf7a0 0x7f38f40c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.504+0000 7f38f3fff640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38f40d14a0 0x7f38f41f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.504+0000 7f38f93bb640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38f415d5b0 0x7f38f406b520 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00c7c0 tx=0x7f38ec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.505+0000 7f38f17fa640 1 -- 172.21.15.5:0/127873893 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec004230 con 0x7f38f415d5b0 2023-11-24T06:16:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.505+0000 7f38f17fa640 1 -- 172.21.15.5:0/127873893 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38ec004390 con 0x7f38f415d5b0 2023-11-24T06:16:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.505+0000 7f38f17fa640 1 -- 172.21.15.5:0/127873893 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38ec007860 con 0x7f38f415d5b0 2023-11-24T06:16:39.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.505+0000 7f38f9bbc640 1 -- 172.21.15.5:0/127873893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f40c8c30 con 0x7f38f415d5b0 2023-11-24T06:16:39.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.505+0000 7f38f9bbc640 1 -- 172.21.15.5:0/127873893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f40c8290 con 0x7f38f415d5b0 2023-11-24T06:16:39.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.506+0000 7f38f9bbc640 1 -- 172.21.15.5:0/127873893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f40d1880 con 0x7f38f415d5b0 2023-11-24T06:16:39.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.507+0000 7f38f17fa640 1 -- 172.21.15.5:0/127873893 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f38ec0079c0 con 0x7f38f415d5b0 2023-11-24T06:16:39.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.508+0000 7f38f17fa640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f38e006ae70 0x7f38e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:39.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.508+0000 7f38f17fa640 1 -- 172.21.15.5:0/127873893 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f38ec089950 con 0x7f38f415d5b0 2023-11-24T06:16:39.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.508+0000 7f38f8bba640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f38e006ae70 0x7f38e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:39.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.509+0000 7f38f8bba640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f38e006ae70 0x7f38e006d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38f4160390 tx=0x7f38e40074d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:39.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.512+0000 7f38f17fa640 1 -- 172.21.15.5:0/127873893 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38ec054a20 con 0x7f38f415d5b0 2023-11-24T06:16:39.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.707+0000 7f38f9bbc640 1 -- 172.21.15.5:0/127873893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38f41f3ce0 con 0x7f38f415d5b0 2023-11-24T06:16:39.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.710+0000 7f38f17fa640 1 -- 172.21.15.5:0/127873893 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16658 (secure 0 0 0) 0x7f38ec089070 con 0x7f38f415d5b0 2023-11-24T06:16:39.713 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:39.713 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:16:33.958286+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:39.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.712+0000 7f38f9bbc640 1 -- 172.21.15.5:0/127873893 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f38e006ae70 msgr2=0x7f38e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:39.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.712+0000 7f38f9bbc640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f38e006ae70 0x7f38e006d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f38f4160390 tx=0x7f38e40074d0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.712+0000 7f38f9bbc640 1 -- 172.21.15.5:0/127873893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38f415d5b0 msgr2=0x7f38f406b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:39.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.712+0000 7f38f9bbc640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38f415d5b0 0x7f38f406b520 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00c7c0 tx=0x7f38ec00cc80 comp rx=0 tx=0).stop 2023-11-24T06:16:39.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.713+0000 7f38f9bbc640 1 -- 172.21.15.5:0/127873893 shutdown_connections 2023-11-24T06:16:39.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.713+0000 7f38f9bbc640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f38e006ae70 0x7f38e006d330 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.713+0000 7f38f9bbc640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f38f40d14a0 0x7f38f41f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.713+0000 7f38f9bbc640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38f415d5b0 0x7f38f406b520 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.713+0000 7f38f9bbc640 1 --2- 172.21.15.5:0/127873893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f38f40cf7a0 0x7f38f40c7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.713+0000 7f38f9bbc640 1 -- 172.21.15.5:0/127873893 >> 172.21.15.5:0/127873893 conn(0x7f38f409c3b0 msgr2=0x7f38f415b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:39.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.713+0000 7f38f9bbc640 1 -- 172.21.15.5:0/127873893 shutdown_connections 2023-11-24T06:16:39.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.713+0000 7f38f9bbc640 1 -- 172.21.15.5:0/127873893 wait complete. 2023-11-24T06:16:39.735 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:39.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:39.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.963+0000 7f9493f90640 1 Processor -- start 2023-11-24T06:16:39.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.963+0000 7f9493f90640 1 -- start start 2023-11-24T06:16:39.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.965+0000 7f9493f90640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f948c06cad0 0x7f948c06e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:39.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.965+0000 7f9493f90640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f948c072280 0x7f948c0b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:39.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.965+0000 7f9493f90640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f948c071780 con 0x7f948c160b10 2023-11-24T06:16:39.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.965+0000 7f9493f90640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f948c0b2320 con 0x7f948c06cad0 2023-11-24T06:16:39.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.965+0000 7f9493f90640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c06df30 con 0x7f948c072280 2023-11-24T06:16:39.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.965+0000 7f949152b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f948c06cad0 0x7f948c06e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.966+0000 7f949152b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f948c06cad0 0x7f948c06e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:46842/0 (socket says 172.21.15.5:46842) 2023-11-24T06:16:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.966+0000 7f949152b640 1 -- 172.21.15.5:0/1127254055 learned_addr learned my addr 172.21.15.5:0/1127254055 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.965+0000 7f949252d640 1 --2- 172.21.15.5:0/1127254055 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f948c072280 0x7f948c0b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.966+0000 7f949252d640 1 -- 172.21.15.5:0/1127254055 >> v2:172.21.15.6:3300/0 conn(0x7f948c06cad0 msgr2=0x7f948c06e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.966+0000 7f949252d640 1 --2- 172.21.15.5:0/1127254055 >> v2:172.21.15.6:3300/0 conn(0x7f948c06cad0 0x7f948c06e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.966+0000 7f949252d640 1 -- 172.21.15.5:0/1127254055 >> v1:172.21.15.6:6789/0 conn(0x7f948c160b10 legacy=0x7f948c06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:39.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.966+0000 7f949252d640 1 -- 172.21.15.5:0/1127254055 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c0711c0 con 0x7f948c072280 2023-11-24T06:16:39.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.967+0000 7f949152b640 1 --2- 172.21.15.5:0/1127254055 >> v2:172.21.15.6:3300/0 conn(0x7f948c06cad0 0x7f948c06e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:16:39.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.968+0000 7f949252d640 1 --2- 172.21.15.5:0/1127254055 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f948c072280 0x7f948c0b2740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9484009d90 tx=0x7f948402be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eea5a86441119fd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:39.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.968+0000 7f9490d2a640 1 -- 172.21.15.5:0/1127254055 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948402cbd0 con 0x7f948c072280 2023-11-24T06:16:39.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.969+0000 7f9490d2a640 1 -- 172.21.15.5:0/1127254055 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948402cd30 con 0x7f948c072280 2023-11-24T06:16:39.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.969+0000 7f9490d2a640 1 -- 172.21.15.5:0/1127254055 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948402cea0 con 0x7f948c072280 2023-11-24T06:16:39.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.969+0000 7f9493f90640 1 -- 172.21.15.5:0/1127254055 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f948c072280 msgr2=0x7f948c0b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:39.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.969+0000 7f9493f90640 1 --2- 172.21.15.5:0/1127254055 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f948c072280 0x7f948c0b2740 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9484009d90 tx=0x7f948402be20 comp rx=0 tx=0).stop 2023-11-24T06:16:39.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.969+0000 7f9493f90640 1 -- 172.21.15.5:0/1127254055 shutdown_connections 2023-11-24T06:16:39.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.969+0000 7f9493f90640 1 --2- 172.21.15.5:0/1127254055 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f948c072280 0x7f948c0b2740 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.969+0000 7f9493f90640 1 --2- 172.21.15.5:0/1127254055 >> v2:172.21.15.6:3300/0 conn(0x7f948c06cad0 0x7f948c06e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.969+0000 7f9493f90640 1 -- 172.21.15.5:0/1127254055 >> 172.21.15.5:0/1127254055 conn(0x7f948c0b0ba0 msgr2=0x7f948c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.970+0000 7f9493f90640 1 -- 172.21.15.5:0/1127254055 shutdown_connections 2023-11-24T06:16:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.970+0000 7f9493f90640 1 -- 172.21.15.5:0/1127254055 wait complete. 2023-11-24T06:16:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.971+0000 7f9493f90640 1 Processor -- start 2023-11-24T06:16:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.972+0000 7f9493f90640 1 -- start start 2023-11-24T06:16:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f9493f90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f948c06cad0 0x7f948c157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f9493f90640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f948c072280 0x7f948c09a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f9493f90640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f948c160b10 0x7f948c09ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:39.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f9493f90640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c09bda0 con 0x7f948c06cad0 2023-11-24T06:16:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f9493f90640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f948c099d50 con 0x7f948c072280 2023-11-24T06:16:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f9493f90640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f948c0c84d0 con 0x7f948c160b10 2023-11-24T06:16:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f9491d2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f948c06cad0 0x7f948c157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f9491d2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f948c06cad0 0x7f948c157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50746/0 (socket says 172.21.15.5:50746) 2023-11-24T06:16:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f9491d2c640 1 -- 172.21.15.5:0/192957577 learned_addr learned my addr 172.21.15.5:0/192957577 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f949152b640 1 --2- 172.21.15.5:0/192957577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f948c072280 0x7f948c09a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f949252d640 1 --2- 172.21.15.5:0/192957577 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f948c160b10 0x7f948c09ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f9491d2c640 1 -- 172.21.15.5:0/192957577 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f948c160b10 msgr2=0x7f948c09ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f9491d2c640 1 --2- 172.21.15.5:0/192957577 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f948c160b10 0x7f948c09ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f9491d2c640 1 -- 172.21.15.5:0/192957577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f948c072280 msgr2=0x7f948c09a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.973+0000 7f9491d2c640 1 --2- 172.21.15.5:0/192957577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f948c072280 0x7f948c09a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.974+0000 7f9491d2c640 1 -- 172.21.15.5:0/192957577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c0094d0 con 0x7f948c06cad0 2023-11-24T06:16:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.974+0000 7f9491d2c640 1 --2- 172.21.15.5:0/192957577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f948c06cad0 0x7f948c157e80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f948800c400 tx=0x7f948800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.974+0000 7f94827fc640 1 -- 172.21.15.5:0/192957577 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94880052d0 con 0x7f948c06cad0 2023-11-24T06:16:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.974+0000 7f94827fc640 1 -- 172.21.15.5:0/192957577 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9488005430 con 0x7f948c06cad0 2023-11-24T06:16:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.974+0000 7f9493f90640 1 -- 172.21.15.5:0/192957577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9484009a70 con 0x7f948c06cad0 2023-11-24T06:16:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.974+0000 7f94827fc640 1 -- 172.21.15.5:0/192957577 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9488003830 con 0x7f948c06cad0 2023-11-24T06:16:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.974+0000 7f9493f90640 1 -- 172.21.15.5:0/192957577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948c09b7e0 con 0x7f948c06cad0 2023-11-24T06:16:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.976+0000 7f9493f90640 1 -- 172.21.15.5:0/192957577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948c09d200 con 0x7f948c06cad0 2023-11-24T06:16:39.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.976+0000 7f94827fc640 1 -- 172.21.15.5:0/192957577 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9488003990 con 0x7f948c06cad0 2023-11-24T06:16:39.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.978+0000 7f94827fc640 1 --2- 172.21.15.5:0/192957577 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f947006ae40 0x7f947006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:39.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.978+0000 7f94827fc640 1 -- 172.21.15.5:0/192957577 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9488088c30 con 0x7f948c06cad0 2023-11-24T06:16:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.978+0000 7f949152b640 1 --2- 172.21.15.5:0/192957577 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f947006ae40 0x7f947006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:39.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.979+0000 7f949152b640 1 --2- 172.21.15.5:0/192957577 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f947006ae40 0x7f947006d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f948c0677e0 tx=0x7f947c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:39.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:39.983+0000 7f94827fc640 1 -- 172.21.15.5:0/192957577 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9488053c40 con 0x7f948c06cad0 2023-11-24T06:16:40.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.179+0000 7f9493f90640 1 -- 172.21.15.5:0/192957577 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f948c09c810 con 0x7f948c06cad0 2023-11-24T06:16:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.180+0000 7f94827fc640 1 -- 172.21.15.5:0/192957577 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94880170a0 con 0x7f948c06cad0 2023-11-24T06:16:40.183 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:40.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.183+0000 7f9493f90640 1 -- 172.21.15.5:0/192957577 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f947006ae40 msgr2=0x7f947006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:40.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.183+0000 7f9493f90640 1 --2- 172.21.15.5:0/192957577 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f947006ae40 0x7f947006d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f948c0677e0 tx=0x7f947c008040 comp rx=0 tx=0).stop 2023-11-24T06:16:40.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.183+0000 7f9493f90640 1 -- 172.21.15.5:0/192957577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f948c06cad0 msgr2=0x7f948c157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:40.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.183+0000 7f9493f90640 1 --2- 172.21.15.5:0/192957577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f948c06cad0 0x7f948c157e80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f948800c400 tx=0x7f948800c8c0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.183+0000 7f9493f90640 1 -- 172.21.15.5:0/192957577 shutdown_connections 2023-11-24T06:16:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.183+0000 7f9493f90640 1 --2- 172.21.15.5:0/192957577 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f947006ae40 0x7f947006d300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.183+0000 7f9493f90640 1 --2- 172.21.15.5:0/192957577 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f948c160b10 0x7f948c09ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.183+0000 7f9493f90640 1 --2- 172.21.15.5:0/192957577 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f948c072280 0x7f948c09a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.183+0000 7f9493f90640 1 --2- 172.21.15.5:0/192957577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f948c06cad0 0x7f948c157e80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.183+0000 7f9493f90640 1 -- 172.21.15.5:0/192957577 >> 172.21.15.5:0/192957577 conn(0x7f948c0b0ba0 msgr2=0x7f948c073310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.184+0000 7f9493f90640 1 -- 172.21.15.5:0/192957577 shutdown_connections 2023-11-24T06:16:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.184+0000 7f9493f90640 1 -- 172.21.15.5:0/192957577 wait complete. 2023-11-24T06:16:40.206 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:40.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.433+0000 7f5c7eed1640 1 Processor -- start 2023-11-24T06:16:40.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.434+0000 7f5c7eed1640 1 -- start start 2023-11-24T06:16:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.435+0000 7f5c7eed1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5c780ad450 0x7f5c78154280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.435+0000 7f5c7eed1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5c78070850 0x7f5c78071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.435+0000 7f5c7eed1640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c780ce650 con 0x7f5c78153ea0 2023-11-24T06:16:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.435+0000 7f5c7eed1640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c780ad8e0 con 0x7f5c78070850 2023-11-24T06:16:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.435+0000 7f5c7eed1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c781547c0 con 0x7f5c780ad450 2023-11-24T06:16:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.435+0000 7f5c7decf640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5c78153ea0 0x7f5c7815d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56336/0 (socket says 172.21.15.5:56336) 2023-11-24T06:16:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.435+0000 7f5c7decf640 1 -- 172.21.15.5:0/3044927255 learned_addr learned my addr 172.21.15.5:0/3044927255 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.435+0000 7f5c7e6d0640 1 --2- 172.21.15.5:0/3044927255 >> v2:172.21.15.5:3300/0 conn(0x7f5c78070850 0x7f5c78071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:40.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.436+0000 7f5c7d6ce640 1 --2- 172.21.15.5:0/3044927255 >> v2:172.21.15.6:3300/0 conn(0x7f5c780ad450 0x7f5c78154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.436+0000 7f5c7cecd640 1 -- 172.21.15.5:0/3044927255 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597637210 0 0) 0x7f5c780ce650 con 0x7f5c78153ea0 2023-11-24T06:16:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.436+0000 7f5c7cecd640 1 -- 172.21.15.5:0/3044927255 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c64003520 con 0x7f5c78153ea0 2023-11-24T06:16:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.436+0000 7f5c7e6d0640 1 -- 172.21.15.5:0/3044927255 >> v2:172.21.15.6:3300/0 conn(0x7f5c780ad450 msgr2=0x7f5c78154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.436+0000 7f5c7e6d0640 1 --2- 172.21.15.5:0/3044927255 >> v2:172.21.15.6:3300/0 conn(0x7f5c780ad450 0x7f5c78154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.436+0000 7f5c7e6d0640 1 -- 172.21.15.5:0/3044927255 >> v1:172.21.15.5:6789/0 conn(0x7f5c78153ea0 legacy=0x7f5c7815d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:40.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.436+0000 7f5c7e6d0640 1 -- 172.21.15.5:0/3044927255 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7815e0a0 con 0x7f5c78070850 2023-11-24T06:16:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.437+0000 7f5c7e6d0640 1 --2- 172.21.15.5:0/3044927255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c78070850 0x7f5c78071350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5c7815dfa0 tx=0x7f5c7002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f694a05b56db31 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.437+0000 7f5c7cecd640 1 -- 172.21.15.5:0/3044927255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7002cc60 con 0x7f5c78070850 2023-11-24T06:16:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.438+0000 7f5c7cecd640 1 -- 172.21.15.5:0/3044927255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7002cdc0 con 0x7f5c78070850 2023-11-24T06:16:40.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.438+0000 7f5c7cecd640 1 -- 172.21.15.5:0/3044927255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70034640 con 0x7f5c78070850 2023-11-24T06:16:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.438+0000 7f5c7eed1640 1 -- 172.21.15.5:0/3044927255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c78070850 msgr2=0x7f5c78071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.438+0000 7f5c7eed1640 1 --2- 172.21.15.5:0/3044927255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c78070850 0x7f5c78071350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5c7815dfa0 tx=0x7f5c7002beb0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.438+0000 7f5c7eed1640 1 -- 172.21.15.5:0/3044927255 shutdown_connections 2023-11-24T06:16:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.438+0000 7f5c7eed1640 1 --2- 172.21.15.5:0/3044927255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c78070850 0x7f5c78071350 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.438+0000 7f5c7eed1640 1 --2- 172.21.15.5:0/3044927255 >> v2:172.21.15.6:3300/0 conn(0x7f5c780ad450 0x7f5c78154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.438+0000 7f5c7eed1640 1 -- 172.21.15.5:0/3044927255 >> 172.21.15.5:0/3044927255 conn(0x7f5c780cec10 msgr2=0x7f5c780cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.438+0000 7f5c7eed1640 1 -- 172.21.15.5:0/3044927255 shutdown_connections 2023-11-24T06:16:40.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.438+0000 7f5c7eed1640 1 -- 172.21.15.5:0/3044927255 wait complete. 2023-11-24T06:16:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.440+0000 7f5c7eed1640 1 Processor -- start 2023-11-24T06:16:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.440+0000 7f5c7eed1640 1 -- start start 2023-11-24T06:16:40.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.441+0000 7f5c7eed1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c78070850 0x7f5c780ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:40.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.441+0000 7f5c7eed1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c780ad450 0x7f5c781582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:40.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.441+0000 7f5c7eed1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c78153ea0 0x7f5c78156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.441+0000 7f5c7eed1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780afca0 con 0x7f5c780ad450 2023-11-24T06:16:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.441+0000 7f5c7eed1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780aee10 con 0x7f5c78070850 2023-11-24T06:16:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.441+0000 7f5c7eed1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780af0f0 con 0x7f5c78153ea0 2023-11-24T06:16:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.442+0000 7f5c7d6ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c780ad450 0x7f5c781582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.442+0000 7f5c7d6ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c780ad450 0x7f5c781582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50768/0 (socket says 172.21.15.5:50768) 2023-11-24T06:16:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.442+0000 7f5c7d6ce640 1 -- 172.21.15.5:0/2808687707 learned_addr learned my addr 172.21.15.5:0/2808687707 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.442+0000 7f5c7decf640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c78070850 0x7f5c780ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.442+0000 7f5c7d6ce640 1 -- 172.21.15.5:0/2808687707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c78153ea0 msgr2=0x7f5c78156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.442+0000 7f5c7e6d0640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c78153ea0 0x7f5c78156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.442+0000 7f5c7d6ce640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c78153ea0 0x7f5c78156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.442+0000 7f5c7d6ce640 1 -- 172.21.15.5:0/2808687707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c78070850 msgr2=0x7f5c780ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.442+0000 7f5c7d6ce640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c78070850 0x7f5c780ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.442+0000 7f5c7d6ce640 1 -- 172.21.15.5:0/2808687707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c70009a70 con 0x7f5c780ad450 2023-11-24T06:16:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.442+0000 7f5c7decf640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c78070850 0x7f5c780ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.442+0000 7f5c7d6ce640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c780ad450 0x7f5c781582f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800d800 tx=0x7f5c6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.443+0000 7f5c6e7fc640 1 -- 172.21.15.5:0/2808687707 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800b6c0 con 0x7f5c780ad450 2023-11-24T06:16:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.443+0000 7f5c6e7fc640 1 -- 172.21.15.5:0/2808687707 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6800b820 con 0x7f5c780ad450 2023-11-24T06:16:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.443+0000 7f5c6e7fc640 1 -- 172.21.15.5:0/2808687707 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800bbb0 con 0x7f5c780ad450 2023-11-24T06:16:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.443+0000 7f5c7eed1640 1 -- 172.21.15.5:0/2808687707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c780af3d0 con 0x7f5c780ad450 2023-11-24T06:16:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.443+0000 7f5c7eed1640 1 -- 172.21.15.5:0/2808687707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c78158830 con 0x7f5c780ad450 2023-11-24T06:16:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.444+0000 7f5c7eed1640 1 -- 172.21.15.5:0/2808687707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7804e840 con 0x7f5c780ad450 2023-11-24T06:16:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.445+0000 7f5c6e7fc640 1 -- 172.21.15.5:0/2808687707 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c6800bd10 con 0x7f5c780ad450 2023-11-24T06:16:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.446+0000 7f5c6e7fc640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5c5c06ae40 0x7f5c5c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.446+0000 7f5c6e7fc640 1 -- 172.21.15.5:0/2808687707 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5c68089ca0 con 0x7f5c780ad450 2023-11-24T06:16:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.446+0000 7f5c7decf640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5c5c06ae40 0x7f5c5c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.447+0000 7f5c7decf640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5c5c06ae40 0x7f5c5c06d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5c74005880 tx=0x7f5c74009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:40.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.451+0000 7f5c6e7fc640 1 -- 172.21.15.5:0/2808687707 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c68054c20 con 0x7f5c780ad450 2023-11-24T06:16:40.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.648+0000 7f5c7eed1640 1 -- 172.21.15.5:0/2808687707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c7815d320 con 0x7f5c780ad450 2023-11-24T06:16:40.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.649+0000 7f5c6e7fc640 1 -- 172.21.15.5:0/2808687707 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5c68058240 con 0x7f5c780ad450 2023-11-24T06:16:40.652 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.651+0000 7f5c7eed1640 1 -- 172.21.15.5:0/2808687707 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5c5c06ae40 msgr2=0x7f5c5c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.651+0000 7f5c7eed1640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5c5c06ae40 0x7f5c5c06d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5c74005880 tx=0x7f5c74009040 comp rx=0 tx=0).stop 2023-11-24T06:16:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.652+0000 7f5c7eed1640 1 -- 172.21.15.5:0/2808687707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c780ad450 msgr2=0x7f5c781582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.652+0000 7f5c7eed1640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c780ad450 0x7f5c781582f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800d800 tx=0x7f5c6800dcc0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.652+0000 7f5c7eed1640 1 -- 172.21.15.5:0/2808687707 shutdown_connections 2023-11-24T06:16:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.652+0000 7f5c7eed1640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5c5c06ae40 0x7f5c5c06d300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.652+0000 7f5c7eed1640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5c78153ea0 0x7f5c78156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.652+0000 7f5c7eed1640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c780ad450 0x7f5c781582f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.652+0000 7f5c7eed1640 1 --2- 172.21.15.5:0/2808687707 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5c78070850 0x7f5c780ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.652+0000 7f5c7eed1640 1 -- 172.21.15.5:0/2808687707 >> 172.21.15.5:0/2808687707 conn(0x7f5c780cec10 msgr2=0x7f5c7815d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.652+0000 7f5c7eed1640 1 -- 172.21.15.5:0/2808687707 shutdown_connections 2023-11-24T06:16:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.652+0000 7f5c7eed1640 1 -- 172.21.15.5:0/2808687707 wait complete. 2023-11-24T06:16:40.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.901+0000 7fe39ffff640 1 Processor -- start 2023-11-24T06:16:40.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.902+0000 7fe39ffff640 1 -- start start 2023-11-24T06:16:40.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.904+0000 7fe39ffff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe3a0153ea0 0x7fe3a00ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:40.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.904+0000 7fe39ffff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a00ce650 con 0x7fe3a0070850 2023-11-24T06:16:40.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.904+0000 7fe39ffff640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a0154710 con 0x7fe3a0154280 2023-11-24T06:16:40.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.904+0000 7fe39ffff640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a015e0a0 con 0x7fe3a0153ea0 2023-11-24T06:16:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.905+0000 7fe39f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe3a0070850 0x7fe3a0071350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56338/0 (socket says 172.21.15.5:56338) 2023-11-24T06:16:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.905+0000 7fe39f7fe640 1 -- 172.21.15.5:0/2632242423 learned_addr learned my addr 172.21.15.5:0/2632242423 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.905+0000 7fe39effd640 1 --2- 172.21.15.5:0/2632242423 >> v2:172.21.15.6:3300/0 conn(0x7fe3a0153ea0 0x7fe3a00ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.905+0000 7fe39e7fc640 1 -- 172.21.15.5:0/2632242423 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726681985 0 0) 0x7fe3a00ce650 con 0x7fe3a0070850 2023-11-24T06:16:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.905+0000 7fe39e7fc640 1 -- 172.21.15.5:0/2632242423 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe38c003520 con 0x7fe3a0070850 2023-11-24T06:16:40.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39e7fc640 1 -- 172.21.15.5:0/2632242423 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3336473713 0 0) 0x7fe38c003520 con 0x7fe3a0070850 2023-11-24T06:16:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39e7fc640 1 -- 172.21.15.5:0/2632242423 >> v2:172.21.15.6:3300/0 conn(0x7fe3a0153ea0 msgr2=0x7fe3a00ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39e7fc640 1 --2- 172.21.15.5:0/2632242423 >> v2:172.21.15.6:3300/0 conn(0x7fe3a0153ea0 0x7fe3a00ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39e7fc640 1 -- 172.21.15.5:0/2632242423 >> v1:172.21.15.6:6789/0 conn(0x7fe3a0154280 legacy=0x7fe3a015d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39e7fc640 1 -- 172.21.15.5:0/2632242423 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a015e380 con 0x7fe3a0070850 2023-11-24T06:16:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39e7fc640 1 -- 172.21.15.5:0/2632242423 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fe3980019f0 con 0x7fe3a0070850 2023-11-24T06:16:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39e7fc640 1 -- 172.21.15.5:0/2632242423 >> v1:172.21.15.5:6789/0 conn(0x7fe3a0070850 legacy=0x7fe3a0071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39e7fc640 1 --2- 172.21.15.5:0/2632242423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c003a30 0x7fe38c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39e7fc640 1 --2- 172.21.15.5:0/2632242423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c0047c0 0x7fe38c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39e7fc640 1 --2- 172.21.15.5:0/2632242423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe38c0051d0 0x7fe38c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39e7fc640 1 -- 172.21.15.5:0/2632242423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a015e0a0 con 0x7fe38c003a30 2023-11-24T06:16:40.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39e7fc640 1 -- 172.21.15.5:0/2632242423 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c00c100 con 0x7fe38c0051d0 2023-11-24T06:16:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39e7fc640 1 -- 172.21.15.5:0/2632242423 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c00c2c0 con 0x7fe38c0047c0 2023-11-24T06:16:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39effd640 1 --2- 172.21.15.5:0/2632242423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c003a30 0x7fe38c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.906+0000 7fe39f7fe640 1 --2- 172.21.15.5:0/2632242423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe38c0051d0 0x7fe38c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.907+0000 7fe397fff640 1 --2- 172.21.15.5:0/2632242423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c0047c0 0x7fe38c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.907+0000 7fe39effd640 1 -- 172.21.15.5:0/2632242423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c0047c0 msgr2=0x7fe38c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.907+0000 7fe39effd640 1 --2- 172.21.15.5:0/2632242423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c0047c0 0x7fe38c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.907+0000 7fe39effd640 1 -- 172.21.15.5:0/2632242423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe38c0051d0 msgr2=0x7fe38c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.907+0000 7fe39effd640 1 --2- 172.21.15.5:0/2632242423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe38c0051d0 0x7fe38c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.907+0000 7fe39effd640 1 -- 172.21.15.5:0/2632242423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c003520 con 0x7fe38c003a30 2023-11-24T06:16:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.907+0000 7fe39f7fe640 1 --2- 172.21.15.5:0/2632242423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe38c0051d0 0x7fe38c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:40.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.907+0000 7fe39f7fe640 1 -- 172.21.15.5:0/2632242423 reap_dead start 2023-11-24T06:16:40.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.907+0000 7fe397fff640 1 --2- 172.21.15.5:0/2632242423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c0047c0 0x7fe38c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.908+0000 7fe39effd640 1 --2- 172.21.15.5:0/2632242423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c003a30 0x7fe38c003e50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe390009ac0 tx=0x7fe38c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65bd79855e275e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.908+0000 7fe39e7fc640 1 -- 172.21.15.5:0/2632242423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39002c6b0 con 0x7fe38c003a30 2023-11-24T06:16:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.908+0000 7fe39e7fc640 1 -- 172.21.15.5:0/2632242423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39002c810 con 0x7fe38c003a30 2023-11-24T06:16:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.908+0000 7fe39e7fc640 1 -- 172.21.15.5:0/2632242423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39002cba0 con 0x7fe38c003a30 2023-11-24T06:16:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.909+0000 7fe39ffff640 1 -- 172.21.15.5:0/2632242423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c003a30 msgr2=0x7fe38c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:40.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.909+0000 7fe39ffff640 1 --2- 172.21.15.5:0/2632242423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c003a30 0x7fe38c003e50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe390009ac0 tx=0x7fe38c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.909+0000 7fe39ffff640 1 -- 172.21.15.5:0/2632242423 shutdown_connections 2023-11-24T06:16:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.909+0000 7fe39ffff640 1 --2- 172.21.15.5:0/2632242423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c003a30 0x7fe38c003e50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.909+0000 7fe39ffff640 1 -- 172.21.15.5:0/2632242423 >> 172.21.15.5:0/2632242423 conn(0x7fe3a00cec10 msgr2=0x7fe3a00cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.909+0000 7fe39ffff640 1 -- 172.21.15.5:0/2632242423 shutdown_connections 2023-11-24T06:16:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.909+0000 7fe39ffff640 1 -- 172.21.15.5:0/2632242423 wait complete. 2023-11-24T06:16:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.911+0000 7fe39ffff640 1 Processor -- start 2023-11-24T06:16:43.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.911+0000 7fe39ffff640 1 -- start start 2023-11-24T06:16:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.912+0000 7fe39ffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c003a30 0x7fe3a013f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.912+0000 7fe39ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3a00b0c20 0x7fe3a0154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.912+0000 7fe39ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a00af880 0x7fe3a00afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.912+0000 7fe39ffff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00b0240 con 0x7fe3a00af880 2023-11-24T06:16:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.912+0000 7fe39ffff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0155e60 con 0x7fe3a00b0c20 2023-11-24T06:16:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.912+0000 7fe39ffff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0155250 con 0x7fe38c003a30 2023-11-24T06:16:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.912+0000 7fe39f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a00af880 0x7fe3a00afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.912+0000 7fe39f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a00af880 0x7fe3a00afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50780/0 (socket says 172.21.15.5:50780) 2023-11-24T06:16:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.913+0000 7fe39f7fe640 1 -- 172.21.15.5:0/1272852243 learned_addr learned my addr 172.21.15.5:0/1272852243 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.913+0000 7fe397fff640 1 --2- 172.21.15.5:0/1272852243 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3a00b0c20 0x7fe3a0154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:43.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.913+0000 7fe39effd640 1 --2- 172.21.15.5:0/1272852243 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c003a30 0x7fe3a013f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.913+0000 7fe39f7fe640 1 -- 172.21.15.5:0/1272852243 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c003a30 msgr2=0x7fe3a013f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.913+0000 7fe39f7fe640 1 --2- 172.21.15.5:0/1272852243 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c003a30 0x7fe3a013f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.913+0000 7fe39f7fe640 1 -- 172.21.15.5:0/1272852243 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3a00b0c20 msgr2=0x7fe3a0154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.913+0000 7fe39f7fe640 1 --2- 172.21.15.5:0/1272852243 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3a00b0c20 0x7fe3a0154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.913+0000 7fe39f7fe640 1 -- 172.21.15.5:0/1272852243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a0155530 con 0x7fe3a00af880 2023-11-24T06:16:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.913+0000 7fe39f7fe640 1 --2- 172.21.15.5:0/1272852243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a00af880 0x7fe3a00afd00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe3a00acbf0 tx=0x7fe39800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.914+0000 7fe3977fe640 1 -- 172.21.15.5:0/1272852243 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39800b8d0 con 0x7fe3a00af880 2023-11-24T06:16:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.914+0000 7fe3977fe640 1 -- 172.21.15.5:0/1272852243 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39800ba30 con 0x7fe3a00af880 2023-11-24T06:16:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.914+0000 7fe39ffff640 1 -- 172.21.15.5:0/1272852243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a0155810 con 0x7fe3a00af880 2023-11-24T06:16:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.914+0000 7fe3977fe640 1 -- 172.21.15.5:0/1272852243 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39800bde0 con 0x7fe3a00af880 2023-11-24T06:16:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.914+0000 7fe39ffff640 1 -- 172.21.15.5:0/1272852243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a0154e40 con 0x7fe3a00af880 2023-11-24T06:16:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.915+0000 7fe39ffff640 1 -- 172.21.15.5:0/1272852243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a0155fd0 con 0x7fe3a00af880 2023-11-24T06:16:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.916+0000 7fe3977fe640 1 -- 172.21.15.5:0/1272852243 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe398007110 con 0x7fe3a00af880 2023-11-24T06:16:43.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.917+0000 7fe3977fe640 1 --2- 172.21.15.5:0/1272852243 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe38006ae70 0x7fe38006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.918+0000 7fe3977fe640 1 -- 172.21.15.5:0/1272852243 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe3980894f0 con 0x7fe3a00af880 2023-11-24T06:16:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.918+0000 7fe39effd640 1 --2- 172.21.15.5:0/1272852243 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe38006ae70 0x7fe38006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.918+0000 7fe39effd640 1 --2- 172.21.15.5:0/1272852243 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe38006ae70 0x7fe38006d330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe390036000 tx=0x7fe390030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:40.922+0000 7fe3977fe640 1 -- 172.21.15.5:0/1272852243 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe398054500 con 0x7fe3a00af880 2023-11-24T06:16:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:41.118+0000 7fe39ffff640 1 -- 172.21.15.5:0/1272852243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3a004e840 con 0x7fe3a00af880 2023-11-24T06:16:43.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:41.118+0000 7fe3977fe640 1 -- 172.21.15.5:0/1272852243 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe398057b20 con 0x7fe3a00af880 2023-11-24T06:16:43.818 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:41.121+0000 7fe39ffff640 1 -- 172.21.15.5:0/1272852243 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe38006ae70 msgr2=0x7fe38006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:41.121+0000 7fe39ffff640 1 --2- 172.21.15.5:0/1272852243 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe38006ae70 0x7fe38006d330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe390036000 tx=0x7fe390030000 comp rx=0 tx=0).stop 2023-11-24T06:16:43.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:41.121+0000 7fe39ffff640 1 -- 172.21.15.5:0/1272852243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a00af880 msgr2=0x7fe3a00afd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:41.121+0000 7fe39ffff640 1 --2- 172.21.15.5:0/1272852243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a00af880 0x7fe3a00afd00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe3a00acbf0 tx=0x7fe39800b0d0 comp rx=0 tx=0).stop 2023-11-24T06:16:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:41.121+0000 7fe39ffff640 1 -- 172.21.15.5:0/1272852243 shutdown_connections 2023-11-24T06:16:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:41.121+0000 7fe39ffff640 1 --2- 172.21.15.5:0/1272852243 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe38006ae70 0x7fe38006d330 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:41.121+0000 7fe39ffff640 1 --2- 172.21.15.5:0/1272852243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a00af880 0x7fe3a00afd00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:41.121+0000 7fe39ffff640 1 --2- 172.21.15.5:0/1272852243 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3a00b0c20 0x7fe3a0154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:41.121+0000 7fe39ffff640 1 --2- 172.21.15.5:0/1272852243 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe38c003a30 0x7fe3a013f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:41.121+0000 7fe39ffff640 1 -- 172.21.15.5:0/1272852243 >> 172.21.15.5:0/1272852243 conn(0x7fe3a00cec10 msgr2=0x7fe3a00ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:41.121+0000 7fe39ffff640 1 -- 172.21.15.5:0/1272852243 shutdown_connections 2023-11-24T06:16:43.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:41.122+0000 7fe39ffff640 1 -- 172.21.15.5:0/1272852243 wait complete. 2023-11-24T06:16:43.821 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:16:44.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.058+0000 7f724bfff640 1 Processor -- start 2023-11-24T06:16:44.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.058+0000 7f724bfff640 1 -- start start 2023-11-24T06:16:44.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.060+0000 7f724bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f724c0b2550 0x7f724c070550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:44.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.060+0000 7f724bfff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724c070a90 con 0x7f724c0b2930 2023-11-24T06:16:44.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.060+0000 7f724bfff640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724c070cf0 con 0x7f724c161700 2023-11-24T06:16:44.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.060+0000 7f724bfff640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f724c070fd0 con 0x7f724c0b2550 2023-11-24T06:16:44.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.060+0000 7f724a7fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f724c0b2930 0x7f724c160ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56146/0 (socket says 172.21.15.5:56146) 2023-11-24T06:16:44.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.060+0000 7f724a7fc640 1 -- 172.21.15.5:0/963411248 learned_addr learned my addr 172.21.15.5:0/963411248 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:44.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.060+0000 7f724affd640 1 --2- 172.21.15.5:0/963411248 >> v2:172.21.15.5:3300/0 conn(0x7f724c0b2550 0x7f724c070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:44.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.061+0000 7f7249ffb640 1 -- 172.21.15.5:0/963411248 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3764328862 0 0) 0x7f724c070a90 con 0x7f724c0b2930 2023-11-24T06:16:44.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.061+0000 7f7249ffb640 1 -- 172.21.15.5:0/963411248 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7238003520 con 0x7f724c0b2930 2023-11-24T06:16:44.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.061+0000 7f724affd640 1 -- 172.21.15.5:0/963411248 >> v1:172.21.15.6:6789/0 conn(0x7f724c161700 legacy=0x7f724c0ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:44.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.061+0000 7f724affd640 1 -- 172.21.15.5:0/963411248 >> v1:172.21.15.5:6789/0 conn(0x7f724c0b2930 legacy=0x7f724c160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:44.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.061+0000 7f724affd640 1 -- 172.21.15.5:0/963411248 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f724c0712b0 con 0x7f724c0b2550 2023-11-24T06:16:44.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.063+0000 7f724affd640 1 --2- 172.21.15.5:0/963411248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c0b2550 0x7f724c070550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f72340097a0 tx=0x7f723402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fbcc55ec5c28ae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:44.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.063+0000 7f7249ffb640 1 -- 172.21.15.5:0/963411248 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234030610 con 0x7f724c0b2550 2023-11-24T06:16:44.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.063+0000 7f7249ffb640 1 -- 172.21.15.5:0/963411248 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7234030770 con 0x7f724c0b2550 2023-11-24T06:16:44.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.063+0000 7f7249ffb640 1 -- 172.21.15.5:0/963411248 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234030b00 con 0x7f724c0b2550 2023-11-24T06:16:44.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.064+0000 7f724bfff640 1 -- 172.21.15.5:0/963411248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c0b2550 msgr2=0x7f724c070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:44.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.064+0000 7f724bfff640 1 --2- 172.21.15.5:0/963411248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c0b2550 0x7f724c070550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f72340097a0 tx=0x7f723402b960 comp rx=0 tx=0).stop 2023-11-24T06:16:44.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.064+0000 7f724bfff640 1 -- 172.21.15.5:0/963411248 shutdown_connections 2023-11-24T06:16:44.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.064+0000 7f724bfff640 1 --2- 172.21.15.5:0/963411248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c0b2550 0x7f724c070550 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:44.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.064+0000 7f724bfff640 1 -- 172.21.15.5:0/963411248 >> 172.21.15.5:0/963411248 conn(0x7f724c0b0840 msgr2=0x7f724c0b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:44.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.064+0000 7f724bfff640 1 -- 172.21.15.5:0/963411248 shutdown_connections 2023-11-24T06:16:44.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.064+0000 7f724bfff640 1 -- 172.21.15.5:0/963411248 wait complete. 2023-11-24T06:16:44.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.066+0000 7f724bfff640 1 Processor -- start 2023-11-24T06:16:44.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.066+0000 7f724bfff640 1 -- start start 2023-11-24T06:16:44.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.067+0000 7f724bfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f724c0b2550 0x7f724c1f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:44.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.067+0000 7f724bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c0b2930 0x7f724c14c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.067+0000 7f724bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724c161700 0x7f724c15c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.067+0000 7f724bfff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c0c3ab0 con 0x7f724c0b2930 2023-11-24T06:16:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.067+0000 7f724bfff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724c1f33e0 con 0x7f724c161700 2023-11-24T06:16:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.067+0000 7f724bfff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724c0c3d00 con 0x7f724c0b2550 2023-11-24T06:16:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.068+0000 7f724a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c0b2930 0x7f724c14c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.068+0000 7f724a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c0b2930 0x7f724c14c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40710/0 (socket says 172.21.15.5:40710) 2023-11-24T06:16:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.068+0000 7f724a7fc640 1 -- 172.21.15.5:0/1922654651 learned_addr learned my addr 172.21.15.5:0/1922654651 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.068+0000 7f724affd640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f724c0b2550 0x7f724c1f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.068+0000 7f724b7fe640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724c161700 0x7f724c15c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.068+0000 7f724a7fc640 1 -- 172.21.15.5:0/1922654651 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f724c0b2550 msgr2=0x7f724c1f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.068+0000 7f724a7fc640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f724c0b2550 0x7f724c1f2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.068+0000 7f724a7fc640 1 -- 172.21.15.5:0/1922654651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724c161700 msgr2=0x7f724c15c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.068+0000 7f724a7fc640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724c161700 0x7f724c15c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:44.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.068+0000 7f724a7fc640 1 -- 172.21.15.5:0/1922654651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7234009400 con 0x7f724c0b2930 2023-11-24T06:16:44.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.068+0000 7f724affd640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f724c0b2550 0x7f724c1f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:44.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.069+0000 7f724a7fc640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c0b2930 0x7f724c14c6a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f723c00c4d0 tx=0x7f723c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:44.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.069+0000 7f72437fe640 1 -- 172.21.15.5:0/1922654651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c005ce0 con 0x7f724c0b2930 2023-11-24T06:16:44.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.069+0000 7f72437fe640 1 -- 172.21.15.5:0/1922654651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723c005e40 con 0x7f724c0b2930 2023-11-24T06:16:44.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.069+0000 7f72437fe640 1 -- 172.21.15.5:0/1922654651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723c007270 con 0x7f724c0b2930 2023-11-24T06:16:44.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.069+0000 7f724bfff640 1 -- 172.21.15.5:0/1922654651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f724c0c3fe0 con 0x7f724c0b2930 2023-11-24T06:16:44.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.069+0000 7f724bfff640 1 -- 172.21.15.5:0/1922654651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f724c0c45a0 con 0x7f724c0b2930 2023-11-24T06:16:44.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.070+0000 7f724bfff640 1 -- 172.21.15.5:0/1922654651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f724c0c4790 con 0x7f724c0b2930 2023-11-24T06:16:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.071+0000 7f72437fe640 1 -- 172.21.15.5:0/1922654651 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f723c0073d0 con 0x7f724c0b2930 2023-11-24T06:16:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.072+0000 7f72437fe640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f722c06ae40 0x7f722c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.073+0000 7f72437fe640 1 -- 172.21.15.5:0/1922654651 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f723c088be0 con 0x7f724c0b2930 2023-11-24T06:16:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.073+0000 7f724affd640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f722c06ae40 0x7f722c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.074+0000 7f724affd640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f722c06ae40 0x7f722c06d300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7234036000 tx=0x7f723403b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:44.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.077+0000 7f72437fe640 1 -- 172.21.15.5:0/1922654651 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f723c053ca0 con 0x7f724c0b2930 2023-11-24T06:16:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.286+0000 7f724bfff640 1 -- 172.21.15.5:0/1922654651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f724c04e840 con 0x7f724c0b2930 2023-11-24T06:16:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.287+0000 7f72437fe640 1 -- 172.21.15.5:0/1922654651 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f723c017020 con 0x7f724c0b2930 2023-11-24T06:16:44.289 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2023-11-24T06:16:44.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.289+0000 7f724bfff640 1 -- 172.21.15.5:0/1922654651 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f722c06ae40 msgr2=0x7f722c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:44.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.289+0000 7f724bfff640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f722c06ae40 0x7f722c06d300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7234036000 tx=0x7f723403b040 comp rx=0 tx=0).stop 2023-11-24T06:16:44.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.289+0000 7f724bfff640 1 -- 172.21.15.5:0/1922654651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c0b2930 msgr2=0x7f724c14c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:44.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.289+0000 7f724bfff640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c0b2930 0x7f724c14c6a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f723c00c4d0 tx=0x7f723c00c990 comp rx=0 tx=0).stop 2023-11-24T06:16:44.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.289+0000 7f724bfff640 1 -- 172.21.15.5:0/1922654651 shutdown_connections 2023-11-24T06:16:44.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.289+0000 7f724bfff640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f722c06ae40 0x7f722c06d300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:44.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.289+0000 7f724bfff640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f724c161700 0x7f724c15c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:44.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.289+0000 7f724bfff640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c0b2930 0x7f724c14c6a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:44.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.289+0000 7f724bfff640 1 --2- 172.21.15.5:0/1922654651 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f724c0b2550 0x7f724c1f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:44.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.289+0000 7f724bfff640 1 -- 172.21.15.5:0/1922654651 >> 172.21.15.5:0/1922654651 conn(0x7f724c0b0840 msgr2=0x7f724c0b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:44.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.289+0000 7f724bfff640 1 -- 172.21.15.5:0/1922654651 shutdown_connections 2023-11-24T06:16:44.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.289+0000 7f724bfff640 1 -- 172.21.15.5:0/1922654651 wait complete. 2023-11-24T06:16:44.308 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi005.front.sepia.ceph.com... 2023-11-24T06:16:44.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:44.309 DEBUG:teuthology.orchestra.run.smithi005:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T06:16:44.405 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:16:44.405 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T06:16:44.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:44.406 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:16:44.476 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:16:44.476 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:16:44.541 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:16:44.541 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:16:44.600 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:44.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.832+0000 7fb49974c640 1 Processor -- start 2023-11-24T06:16:44.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.832+0000 7fb49974c640 1 -- start start 2023-11-24T06:16:44.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.833+0000 7fb49974c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb49415d460 0x7fb4940b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:44.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.833+0000 7fb49974c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb4940d0230 0x7fb4940d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:44.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.833+0000 7fb49974c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4940d0ba0 con 0x7fb4940d1300 2023-11-24T06:16:44.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.833+0000 7fb49974c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49415d840 con 0x7fb4940d0230 2023-11-24T06:16:44.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.833+0000 7fb49974c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940d1760 con 0x7fb49415d460 2023-11-24T06:16:44.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.833+0000 7fb4927fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb4940d1300 0x7fb4940b8c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56154/0 (socket says 172.21.15.5:56154) 2023-11-24T06:16:44.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.833+0000 7fb4927fc640 1 -- 172.21.15.5:0/1921988224 learned_addr learned my addr 172.21.15.5:0/1921988224 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:44.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.834+0000 7fb492ffd640 1 --2- 172.21.15.5:0/1921988224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb49415d460 0x7fb4940b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:44.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.834+0000 7fb4937fe640 1 --2- 172.21.15.5:0/1921988224 >> v2:172.21.15.6:3300/0 conn(0x7fb4940d0230 0x7fb4940d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:44.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.834+0000 7fb491ffb640 1 -- 172.21.15.5:0/1921988224 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827619888 0 0) 0x7fb4940d0ba0 con 0x7fb4940d1300 2023-11-24T06:16:44.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.834+0000 7fb491ffb640 1 -- 172.21.15.5:0/1921988224 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb480003520 con 0x7fb4940d1300 2023-11-24T06:16:44.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.834+0000 7fb491ffb640 1 -- 172.21.15.5:0/1921988224 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1812190609 0 0) 0x7fb480003520 con 0x7fb4940d1300 2023-11-24T06:16:44.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.834+0000 7fb491ffb640 1 -- 172.21.15.5:0/1921988224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb49415d460 msgr2=0x7fb4940b62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:44.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.834+0000 7fb491ffb640 1 --2- 172.21.15.5:0/1921988224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb49415d460 0x7fb4940b62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:44.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.834+0000 7fb491ffb640 1 -- 172.21.15.5:0/1921988224 >> v2:172.21.15.6:3300/0 conn(0x7fb4940d0230 msgr2=0x7fb4940d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:44.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.834+0000 7fb491ffb640 1 --2- 172.21.15.5:0/1921988224 >> v2:172.21.15.6:3300/0 conn(0x7fb4940d0230 0x7fb4940d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:44.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.834+0000 7fb491ffb640 1 -- 172.21.15.5:0/1921988224 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4940b69a0 con 0x7fb4940d1300 2023-11-24T06:16:44.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.834+0000 7fb491ffb640 1 -- 172.21.15.5:0/1921988224 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fb47c001670 con 0x7fb4940d1300 2023-11-24T06:16:44.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.835+0000 7fb491ffb640 1 -- 172.21.15.5:0/1921988224 >> v1:172.21.15.5:6789/0 conn(0x7fb4940d1300 legacy=0x7fb4940b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:44.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.835+0000 7fb491ffb640 1 --2- 172.21.15.5:0/1921988224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb480003970 0x7fb480003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:44.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.835+0000 7fb491ffb640 1 --2- 172.21.15.5:0/1921988224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4800047c0 0x7fb480004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:44.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.835+0000 7fb491ffb640 1 --2- 172.21.15.5:0/1921988224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb49415d460 0x7fb480005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:44.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.835+0000 7fb491ffb640 1 -- 172.21.15.5:0/1921988224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49415d840 con 0x7fb4800047c0 2023-11-24T06:16:44.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.835+0000 7fb491ffb640 1 -- 172.21.15.5:0/1921988224 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940d1760 con 0x7fb480003970 2023-11-24T06:16:44.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.835+0000 7fb491ffb640 1 -- 172.21.15.5:0/1921988224 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48000b980 con 0x7fb49415d460 2023-11-24T06:16:44.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.835+0000 7fb4927fc640 1 --2- 172.21.15.5:0/1921988224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4800047c0 0x7fb480004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:44.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.835+0000 7fb492ffd640 1 --2- 172.21.15.5:0/1921988224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb480003970 0x7fb480003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:44.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.835+0000 7fb4937fe640 1 --2- 172.21.15.5:0/1921988224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb49415d460 0x7fb480005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:44.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.836+0000 7fb4927fc640 1 -- 172.21.15.5:0/1921988224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb49415d460 msgr2=0x7fb480005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:44.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.836+0000 7fb4927fc640 1 --2- 172.21.15.5:0/1921988224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb49415d460 0x7fb480005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:44.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.836+0000 7fb4927fc640 1 -- 172.21.15.5:0/1921988224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb480003970 msgr2=0x7fb480003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:44.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.836+0000 7fb4927fc640 1 --2- 172.21.15.5:0/1921988224 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb480003970 0x7fb480003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:44.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.836+0000 7fb4927fc640 1 -- 172.21.15.5:0/1921988224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb480003520 con 0x7fb4800047c0 2023-11-24T06:16:44.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.836+0000 7fb4937fe640 1 --2- 172.21.15.5:0/1921988224 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb49415d460 0x7fb480005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.837+0000 7fb4927fc640 1 --2- 172.21.15.5:0/1921988224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4800047c0 0x7fb480004c40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00c240 tx=0x7fb4800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd59cd7fb05a27b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.837+0000 7fb491ffb640 1 -- 172.21.15.5:0/1921988224 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c02cb00 con 0x7fb4800047c0 2023-11-24T06:16:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.837+0000 7fb491ffb640 1 -- 172.21.15.5:0/1921988224 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47c02cc60 con 0x7fb4800047c0 2023-11-24T06:16:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.837+0000 7fb491ffb640 1 -- 172.21.15.5:0/1921988224 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c005af0 con 0x7fb4800047c0 2023-11-24T06:16:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.837+0000 7fb49974c640 1 -- 172.21.15.5:0/1921988224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4800047c0 msgr2=0x7fb480004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.837+0000 7fb49974c640 1 --2- 172.21.15.5:0/1921988224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4800047c0 0x7fb480004c40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00c240 tx=0x7fb4800057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.838+0000 7fb4937fe640 1 -- 172.21.15.5:0/1921988224 reap_dead start 2023-11-24T06:16:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.838+0000 7fb49974c640 1 -- 172.21.15.5:0/1921988224 shutdown_connections 2023-11-24T06:16:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.838+0000 7fb49974c640 1 -- 172.21.15.5:0/1921988224 >> 172.21.15.5:0/1921988224 conn(0x7fb494071f90 msgr2=0x7fb4940723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.838+0000 7fb49974c640 1 -- 172.21.15.5:0/1921988224 shutdown_connections 2023-11-24T06:16:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.838+0000 7fb49974c640 1 -- 172.21.15.5:0/1921988224 wait complete. 2023-11-24T06:16:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.840+0000 7fb49974c640 1 Processor -- start 2023-11-24T06:16:44.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.840+0000 7fb49974c640 1 -- start start 2023-11-24T06:16:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb49974c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4800047c0 0x7fb49415f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb49974c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb494151a80 0x7fb4940d2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:44.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb49974c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4940d2ec0 0x7fb49415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb49974c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4941f3fa0 con 0x7fb4800047c0 2023-11-24T06:16:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb49974c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4940d3340 con 0x7fb4940d2ec0 2023-11-24T06:16:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb49974c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4941f3320 con 0x7fb494151a80 2023-11-24T06:16:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb492ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4800047c0 0x7fb49415f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb492ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4800047c0 0x7fb49415f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40734/0 (socket says 172.21.15.5:40734) 2023-11-24T06:16:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb492ffd640 1 -- 172.21.15.5:0/1491484256 learned_addr learned my addr 172.21.15.5:0/1491484256 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb4927fc640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb494151a80 0x7fb4940d2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb4937fe640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4940d2ec0 0x7fb49415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb492ffd640 1 -- 172.21.15.5:0/1491484256 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb494151a80 msgr2=0x7fb4940d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:44.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb492ffd640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb494151a80 0x7fb4940d2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb492ffd640 1 -- 172.21.15.5:0/1491484256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4940d2ec0 msgr2=0x7fb49415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb492ffd640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4940d2ec0 0x7fb49415d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb492ffd640 1 -- 172.21.15.5:0/1491484256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4941f3600 con 0x7fb4800047c0 2023-11-24T06:16:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.841+0000 7fb4927fc640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb494151a80 0x7fb4940d2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.842+0000 7fb492ffd640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4800047c0 0x7fb49415f650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb484009fb0 tx=0x7fb4840115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.842+0000 7fb48b7fe640 1 -- 172.21.15.5:0/1491484256 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400e030 con 0x7fb4800047c0 2023-11-24T06:16:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.842+0000 7fb48b7fe640 1 -- 172.21.15.5:0/1491484256 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb484016470 con 0x7fb4800047c0 2023-11-24T06:16:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.842+0000 7fb48b7fe640 1 -- 172.21.15.5:0/1491484256 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb484016820 con 0x7fb4800047c0 2023-11-24T06:16:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.842+0000 7fb49974c640 1 -- 172.21.15.5:0/1491484256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4941f38e0 con 0x7fb4800047c0 2023-11-24T06:16:44.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.842+0000 7fb49974c640 1 -- 172.21.15.5:0/1491484256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49415fb90 con 0x7fb4800047c0 2023-11-24T06:16:44.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.844+0000 7fb49974c640 1 -- 172.21.15.5:0/1491484256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb494074770 con 0x7fb4800047c0 2023-11-24T06:16:44.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.844+0000 7fb48b7fe640 1 -- 172.21.15.5:0/1491484256 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb484016980 con 0x7fb4800047c0 2023-11-24T06:16:44.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.845+0000 7fb48b7fe640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb47406ae70 0x7fb47406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:44.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.845+0000 7fb48b7fe640 1 -- 172.21.15.5:0/1491484256 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb48400f070 con 0x7fb4800047c0 2023-11-24T06:16:44.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.845+0000 7fb4927fc640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb47406ae70 0x7fb47406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:44.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.846+0000 7fb4927fc640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb47406ae70 0x7fb47406d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb47c034000 tx=0x7fb47c007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:44.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:44.850+0000 7fb48b7fe640 1 -- 172.21.15.5:0/1491484256 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4840554c0 con 0x7fb4800047c0 2023-11-24T06:16:45.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.048+0000 7fb49974c640 1 -- 172.21.15.5:0/1491484256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4940c65e0 con 0x7fb4800047c0 2023-11-24T06:16:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.051+0000 7fb48b7fe640 1 -- 172.21.15.5:0/1491484256 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16658 (secure 0 0 0) 0x7fb484058ae0 con 0x7fb4800047c0 2023-11-24T06:16:45.054 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:45.054 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:16:33.958286+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.054+0000 7fb49974c640 1 -- 172.21.15.5:0/1491484256 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb47406ae70 msgr2=0x7fb47406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.054+0000 7fb49974c640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb47406ae70 0x7fb47406d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb47c034000 tx=0x7fb47c007000 comp rx=0 tx=0).stop 2023-11-24T06:16:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.054+0000 7fb49974c640 1 -- 172.21.15.5:0/1491484256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4800047c0 msgr2=0x7fb49415f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.054+0000 7fb49974c640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4800047c0 0x7fb49415f650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb484009fb0 tx=0x7fb4840115a0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.054+0000 7fb49974c640 1 -- 172.21.15.5:0/1491484256 shutdown_connections 2023-11-24T06:16:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.054+0000 7fb49974c640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb47406ae70 0x7fb47406d330 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.054+0000 7fb49974c640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb494151a80 0x7fb4940d2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.054+0000 7fb49974c640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4940d2ec0 0x7fb49415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.054+0000 7fb49974c640 1 --2- 172.21.15.5:0/1491484256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4800047c0 0x7fb49415f650 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.054+0000 7fb49974c640 1 -- 172.21.15.5:0/1491484256 >> 172.21.15.5:0/1491484256 conn(0x7fb494071f90 msgr2=0x7fb4940b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.054+0000 7fb49974c640 1 -- 172.21.15.5:0/1491484256 shutdown_connections 2023-11-24T06:16:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.054+0000 7fb49974c640 1 -- 172.21.15.5:0/1491484256 wait complete. 2023-11-24T06:16:45.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:45.080 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:45.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.306+0000 7f5615e78640 1 Processor -- start 2023-11-24T06:16:45.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.307+0000 7f5615e78640 1 -- start start 2023-11-24T06:16:45.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.308+0000 7f5615e78640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5610144770 0x7f561015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:45.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.308+0000 7f5615e78640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f56100b35a0 0x7f561015f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:45.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.308+0000 7f5615e78640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561015f930 con 0x7f561006c9d0 2023-11-24T06:16:45.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.308+0000 7f5615e78640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56100b43b0 con 0x7f5610144770 2023-11-24T06:16:45.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.308+0000 7f5615e78640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56100b3b10 con 0x7f56100b35a0 2023-11-24T06:16:45.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.309+0000 7f5614e76640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5610144770 0x7f561015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:45.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.309+0000 7f5614e76640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5610144770 0x7f561015eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40742/0 (socket says 172.21.15.5:40742) 2023-11-24T06:16:45.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.309+0000 7f5614e76640 1 -- 172.21.15.5:0/31058228 learned_addr learned my addr 172.21.15.5:0/31058228 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:45.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.309+0000 7f560ffff640 1 --2- 172.21.15.5:0/31058228 >> v2:172.21.15.6:3300/0 conn(0x7f56100b35a0 0x7f561015f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:45.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.309+0000 7f5614e76640 1 -- 172.21.15.5:0/31058228 >> v2:172.21.15.6:3300/0 conn(0x7f56100b35a0 msgr2=0x7f561015f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:45.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.309+0000 7f5614e76640 1 --2- 172.21.15.5:0/31058228 >> v2:172.21.15.6:3300/0 conn(0x7f56100b35a0 0x7f561015f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.309+0000 7f5614e76640 1 -- 172.21.15.5:0/31058228 >> v1:172.21.15.6:6789/0 conn(0x7f561006c9d0 legacy=0x7f561006ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:45.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.309+0000 7f5614e76640 1 -- 172.21.15.5:0/31058228 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56100b3df0 con 0x7f5610144770 2023-11-24T06:16:45.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.311+0000 7f5614e76640 1 --2- 172.21.15.5:0/31058228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5610144770 0x7f561015eb60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f56100ae900 tx=0x7f560002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4435438a78f6b219 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:45.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.311+0000 7f560f7fe640 1 -- 172.21.15.5:0/31058228 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560002ca80 con 0x7f5610144770 2023-11-24T06:16:45.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.311+0000 7f560f7fe640 1 -- 172.21.15.5:0/31058228 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560002cbe0 con 0x7f5610144770 2023-11-24T06:16:45.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.311+0000 7f560f7fe640 1 -- 172.21.15.5:0/31058228 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56000343f0 con 0x7f5610144770 2023-11-24T06:16:45.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.312+0000 7f5615e78640 1 -- 172.21.15.5:0/31058228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5610144770 msgr2=0x7f561015eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.312+0000 7f5615e78640 1 --2- 172.21.15.5:0/31058228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5610144770 0x7f561015eb60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f56100ae900 tx=0x7f560002bee0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.312+0000 7f5615e78640 1 -- 172.21.15.5:0/31058228 shutdown_connections 2023-11-24T06:16:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.312+0000 7f5615e78640 1 --2- 172.21.15.5:0/31058228 >> v2:172.21.15.6:3300/0 conn(0x7f56100b35a0 0x7f561015f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.312+0000 7f5615e78640 1 --2- 172.21.15.5:0/31058228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5610144770 0x7f561015eb60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.312+0000 7f5615e78640 1 -- 172.21.15.5:0/31058228 >> 172.21.15.5:0/31058228 conn(0x7f56100cec10 msgr2=0x7f56100cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.312+0000 7f5615e78640 1 -- 172.21.15.5:0/31058228 shutdown_connections 2023-11-24T06:16:45.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.312+0000 7f5615e78640 1 -- 172.21.15.5:0/31058228 wait complete. 2023-11-24T06:16:45.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.315+0000 7f5615e78640 1 Processor -- start 2023-11-24T06:16:45.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.315+0000 7f5615e78640 1 -- start start 2023-11-24T06:16:45.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.316+0000 7f5615e78640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f561006c9d0 0x7f56101588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:45.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.316+0000 7f5615e78640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56100b35a0 0x7f56100afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:45.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.316+0000 7f5615e78640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5610144770 0x7f56100a92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:45.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.316+0000 7f5615e78640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56100a8500 con 0x7f5610144770 2023-11-24T06:16:45.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.316+0000 7f5615e78640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56100a87e0 con 0x7f56100b35a0 2023-11-24T06:16:45.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.316+0000 7f5615e78640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56100a8ac0 con 0x7f561006c9d0 2023-11-24T06:16:45.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.317+0000 7f5615677640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5610144770 0x7f56100a92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:45.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.317+0000 7f5615677640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5610144770 0x7f56100a92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40758/0 (socket says 172.21.15.5:40758) 2023-11-24T06:16:45.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.317+0000 7f5615677640 1 -- 172.21.15.5:0/1854429793 learned_addr learned my addr 172.21.15.5:0/1854429793 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:45.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.317+0000 7f5614e76640 1 --2- 172.21.15.5:0/1854429793 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f561006c9d0 0x7f56101588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:45.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.317+0000 7f560ffff640 1 --2- 172.21.15.5:0/1854429793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56100b35a0 0x7f56100afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.317+0000 7f5615677640 1 -- 172.21.15.5:0/1854429793 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f561006c9d0 msgr2=0x7f56101588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.317+0000 7f5615677640 1 --2- 172.21.15.5:0/1854429793 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f561006c9d0 0x7f56101588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.317+0000 7f5615677640 1 -- 172.21.15.5:0/1854429793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56100b35a0 msgr2=0x7f56100afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.317+0000 7f5615677640 1 --2- 172.21.15.5:0/1854429793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56100b35a0 0x7f56100afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.317+0000 7f5615677640 1 -- 172.21.15.5:0/1854429793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5600009b30 con 0x7f5610144770 2023-11-24T06:16:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.318+0000 7f5615677640 1 --2- 172.21.15.5:0/1854429793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5610144770 0x7f56100a92c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f560800c410 tx=0x7f560800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.318+0000 7f560d7fa640 1 -- 172.21.15.5:0/1854429793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608011790 con 0x7f5610144770 2023-11-24T06:16:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.318+0000 7f560d7fa640 1 -- 172.21.15.5:0/1854429793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56080118f0 con 0x7f5610144770 2023-11-24T06:16:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.318+0000 7f560d7fa640 1 -- 172.21.15.5:0/1854429793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5608011ce0 con 0x7f5610144770 2023-11-24T06:16:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.318+0000 7f5615e78640 1 -- 172.21.15.5:0/1854429793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56100a8da0 con 0x7f5610144770 2023-11-24T06:16:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.318+0000 7f5615e78640 1 -- 172.21.15.5:0/1854429793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56100aac40 con 0x7f5610144770 2023-11-24T06:16:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.319+0000 7f5615e78640 1 -- 172.21.15.5:0/1854429793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56100a8f90 con 0x7f5610144770 2023-11-24T06:16:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.320+0000 7f560d7fa640 1 -- 172.21.15.5:0/1854429793 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f560801c460 con 0x7f5610144770 2023-11-24T06:16:45.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.321+0000 7f560d7fa640 1 --2- 172.21.15.5:0/1854429793 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f55fc06ae40 0x7f55fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:45.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.321+0000 7f560d7fa640 1 -- 172.21.15.5:0/1854429793 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5608089300 con 0x7f5610144770 2023-11-24T06:16:45.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.322+0000 7f5614e76640 1 --2- 172.21.15.5:0/1854429793 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f55fc06ae40 0x7f55fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:45.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.322+0000 7f5614e76640 1 --2- 172.21.15.5:0/1854429793 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f55fc06ae40 0x7f55fc06d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5600035000 tx=0x7f5600030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.326+0000 7f560d7fa640 1 -- 172.21.15.5:0/1854429793 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5608054310 con 0x7f5610144770 2023-11-24T06:16:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.522+0000 7f5615e78640 1 -- 172.21.15.5:0/1854429793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56100a8640 con 0x7f5610144770 2023-11-24T06:16:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.523+0000 7f560d7fa640 1 -- 172.21.15.5:0/1854429793 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f56100a8640 con 0x7f5610144770 2023-11-24T06:16:45.526 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.526+0000 7f5615e78640 1 -- 172.21.15.5:0/1854429793 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f55fc06ae40 msgr2=0x7f55fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.526+0000 7f5615e78640 1 --2- 172.21.15.5:0/1854429793 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f55fc06ae40 0x7f55fc06d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5600035000 tx=0x7f5600030000 comp rx=0 tx=0).stop 2023-11-24T06:16:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.526+0000 7f5615e78640 1 -- 172.21.15.5:0/1854429793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5610144770 msgr2=0x7f56100a92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.526+0000 7f5615e78640 1 --2- 172.21.15.5:0/1854429793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5610144770 0x7f56100a92c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f560800c410 tx=0x7f560800c8d0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.526+0000 7f5615e78640 1 -- 172.21.15.5:0/1854429793 shutdown_connections 2023-11-24T06:16:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.526+0000 7f5615e78640 1 --2- 172.21.15.5:0/1854429793 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f55fc06ae40 0x7f55fc06d300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.526+0000 7f5615e78640 1 --2- 172.21.15.5:0/1854429793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5610144770 0x7f56100a92c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.526+0000 7f5615e78640 1 --2- 172.21.15.5:0/1854429793 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56100b35a0 0x7f56100afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.526+0000 7f5615e78640 1 --2- 172.21.15.5:0/1854429793 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f561006c9d0 0x7f56101588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.526+0000 7f5615e78640 1 -- 172.21.15.5:0/1854429793 >> 172.21.15.5:0/1854429793 conn(0x7f56100cec10 msgr2=0x7f561015d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.526+0000 7f5615e78640 1 -- 172.21.15.5:0/1854429793 shutdown_connections 2023-11-24T06:16:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.526+0000 7f5615e78640 1 -- 172.21.15.5:0/1854429793 wait complete. 2023-11-24T06:16:45.549 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:45.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.757+0000 7f5193fff640 1 Processor -- start 2023-11-24T06:16:45.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.758+0000 7f5193fff640 1 -- start start 2023-11-24T06:16:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.760+0000 7f5193fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f51941435d0 0x7f5194071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.760+0000 7f5193fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51940b4140 0x7f5194070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.760+0000 7f5193fff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5194070d40 con 0x7f51941439b0 2023-11-24T06:16:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.760+0000 7f5193fff640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5194071020 con 0x7f51941435d0 2023-11-24T06:16:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.760+0000 7f5193fff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5194071300 con 0x7f51940b4140 2023-11-24T06:16:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.760+0000 7f5192ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f51941435d0 0x7f5194071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.760+0000 7f5192ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f51941435d0 0x7f5194071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40764/0 (socket says 172.21.15.5:40764) 2023-11-24T06:16:45.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.760+0000 7f518a7fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f51941439b0 0x7f51940b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56170/0 (socket says 172.21.15.5:56170) 2023-11-24T06:16:45.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.760+0000 7f5192ffd640 1 -- 172.21.15.5:0/1294517441 learned_addr learned my addr 172.21.15.5:0/1294517441 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:45.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.761+0000 7f51937fe640 1 --2- 172.21.15.5:0/1294517441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51940b4140 0x7f5194070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:45.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.761+0000 7f5192ffd640 1 -- 172.21.15.5:0/1294517441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51940b4140 msgr2=0x7f5194070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:45.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.761+0000 7f5192ffd640 1 --2- 172.21.15.5:0/1294517441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51940b4140 0x7f5194070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.761+0000 7f5192ffd640 1 -- 172.21.15.5:0/1294517441 >> v1:172.21.15.5:6789/0 conn(0x7f51941439b0 legacy=0x7f51940b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:45.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.761+0000 7f5192ffd640 1 -- 172.21.15.5:0/1294517441 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51940715e0 con 0x7f51941435d0 2023-11-24T06:16:45.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.762+0000 7f5192ffd640 1 --2- 172.21.15.5:0/1294517441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51941435d0 0x7f5194071de0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5184009e50 tx=0x7f518402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb217782180f7aa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:45.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.762+0000 7f51927fc640 1 -- 172.21.15.5:0/1294517441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518402cb00 con 0x7f51941435d0 2023-11-24T06:16:45.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.762+0000 7f51927fc640 1 -- 172.21.15.5:0/1294517441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518402cc60 con 0x7f51941435d0 2023-11-24T06:16:45.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.762+0000 7f51927fc640 1 -- 172.21.15.5:0/1294517441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51840343f0 con 0x7f51941435d0 2023-11-24T06:16:45.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.763+0000 7f5193fff640 1 -- 172.21.15.5:0/1294517441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51941435d0 msgr2=0x7f5194071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:45.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.763+0000 7f5193fff640 1 --2- 172.21.15.5:0/1294517441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51941435d0 0x7f5194071de0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5184009e50 tx=0x7f518402bf60 comp rx=0 tx=0).stop 2023-11-24T06:16:45.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.763+0000 7f5193fff640 1 -- 172.21.15.5:0/1294517441 shutdown_connections 2023-11-24T06:16:45.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.763+0000 7f5193fff640 1 --2- 172.21.15.5:0/1294517441 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51940b4140 0x7f5194070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.763+0000 7f5193fff640 1 --2- 172.21.15.5:0/1294517441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51941435d0 0x7f5194071de0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.763+0000 7f5193fff640 1 -- 172.21.15.5:0/1294517441 >> 172.21.15.5:0/1294517441 conn(0x7f51940cec10 msgr2=0x7f51940cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:45.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.763+0000 7f5193fff640 1 -- 172.21.15.5:0/1294517441 shutdown_connections 2023-11-24T06:16:45.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.763+0000 7f5193fff640 1 -- 172.21.15.5:0/1294517441 wait complete. 2023-11-24T06:16:45.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.765+0000 7f5193fff640 1 Processor -- start 2023-11-24T06:16:45.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.765+0000 7f5193fff640 1 -- start start 2023-11-24T06:16:45.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.766+0000 7f5193fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51940b4140 0x7f51940a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:45.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.766+0000 7f5193fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51941435d0 0x7f51941595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:45.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.766+0000 7f5193fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f51941439b0 0x7f51940cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:45.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.766+0000 7f5193fff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5194140bd0 con 0x7f51940b4140 2023-11-24T06:16:45.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.766+0000 7f5193fff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519413ffc0 con 0x7f51941439b0 2023-11-24T06:16:45.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.766+0000 7f5193fff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51941402a0 con 0x7f51941435d0 2023-11-24T06:16:45.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.766+0000 7f5192ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51940b4140 0x7f51940a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:45.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.766+0000 7f5192ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51940b4140 0x7f51940a8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40778/0 (socket says 172.21.15.5:40778) 2023-11-24T06:16:45.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.766+0000 7f5192ffd640 1 -- 172.21.15.5:0/1786560798 learned_addr learned my addr 172.21.15.5:0/1786560798 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.767+0000 7f518a7fc640 1 --2- 172.21.15.5:0/1786560798 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51941435d0 0x7f51941595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.767+0000 7f5192ffd640 1 -- 172.21.15.5:0/1786560798 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51941435d0 msgr2=0x7f51941595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.767+0000 7f5192ffd640 1 --2- 172.21.15.5:0/1786560798 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51941435d0 0x7f51941595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.767+0000 7f5192ffd640 1 -- 172.21.15.5:0/1786560798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f51941439b0 msgr2=0x7f51940cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.767+0000 7f51937fe640 1 --2- 172.21.15.5:0/1786560798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f51941439b0 0x7f51940cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.767+0000 7f5192ffd640 1 --2- 172.21.15.5:0/1786560798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f51941439b0 0x7f51940cd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.767+0000 7f5192ffd640 1 -- 172.21.15.5:0/1786560798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5184009b30 con 0x7f51940b4140 2023-11-24T06:16:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.767+0000 7f5192ffd640 1 --2- 172.21.15.5:0/1786560798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51940b4140 0x7f51940a8e70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5184000f80 tx=0x7f5184004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.768+0000 7f518bfff640 1 -- 172.21.15.5:0/1786560798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184009510 con 0x7f51940b4140 2023-11-24T06:16:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.768+0000 7f518bfff640 1 -- 172.21.15.5:0/1786560798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518403e450 con 0x7f51940b4140 2023-11-24T06:16:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.768+0000 7f518bfff640 1 -- 172.21.15.5:0/1786560798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518403e800 con 0x7f51940b4140 2023-11-24T06:16:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.768+0000 7f5193fff640 1 -- 172.21.15.5:0/1786560798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5194140580 con 0x7f51940b4140 2023-11-24T06:16:45.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.768+0000 7f5193fff640 1 -- 172.21.15.5:0/1786560798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51940ce7e0 con 0x7f51940b4140 2023-11-24T06:16:45.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.769+0000 7f5193fff640 1 -- 172.21.15.5:0/1786560798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519404e840 con 0x7f51940b4140 2023-11-24T06:16:45.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.770+0000 7f518bfff640 1 -- 172.21.15.5:0/1786560798 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f518403e960 con 0x7f51940b4140 2023-11-24T06:16:45.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.771+0000 7f518bfff640 1 --2- 172.21.15.5:0/1786560798 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f517406ae70 0x7f517406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:45.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.771+0000 7f518bfff640 1 -- 172.21.15.5:0/1786560798 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f51840ac460 con 0x7f51940b4140 2023-11-24T06:16:45.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.771+0000 7f518a7fc640 1 --2- 172.21.15.5:0/1786560798 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f517406ae70 0x7f517406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:45.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.772+0000 7f518a7fc640 1 --2- 172.21.15.5:0/1786560798 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f517406ae70 0x7f517406d330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f517c007fd0 tx=0x7f517c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:45.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.775+0000 7f518bfff640 1 -- 172.21.15.5:0/1786560798 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5184077480 con 0x7f51940b4140 2023-11-24T06:16:45.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.965+0000 7f5193fff640 1 -- 172.21.15.5:0/1786560798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f519413ef80 con 0x7f51940b4140 2023-11-24T06:16:45.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.966+0000 7f518bfff640 1 -- 172.21.15.5:0/1786560798 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f518407aaa0 con 0x7f51940b4140 2023-11-24T06:16:45.969 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:45.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.968+0000 7f5193fff640 1 -- 172.21.15.5:0/1786560798 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f517406ae70 msgr2=0x7f517406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:45.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.968+0000 7f5193fff640 1 --2- 172.21.15.5:0/1786560798 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f517406ae70 0x7f517406d330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f517c007fd0 tx=0x7f517c009040 comp rx=0 tx=0).stop 2023-11-24T06:16:45.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.969+0000 7f5193fff640 1 -- 172.21.15.5:0/1786560798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51940b4140 msgr2=0x7f51940a8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:45.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.969+0000 7f5193fff640 1 --2- 172.21.15.5:0/1786560798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51940b4140 0x7f51940a8e70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5184000f80 tx=0x7f5184004a40 comp rx=0 tx=0).stop 2023-11-24T06:16:45.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.969+0000 7f5193fff640 1 -- 172.21.15.5:0/1786560798 shutdown_connections 2023-11-24T06:16:45.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.969+0000 7f5193fff640 1 --2- 172.21.15.5:0/1786560798 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f517406ae70 0x7f517406d330 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.969+0000 7f5193fff640 1 --2- 172.21.15.5:0/1786560798 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f51941435d0 0x7f51941595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.969+0000 7f5193fff640 1 --2- 172.21.15.5:0/1786560798 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f51941439b0 0x7f51940cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.969+0000 7f5193fff640 1 --2- 172.21.15.5:0/1786560798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51940b4140 0x7f51940a8e70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.969+0000 7f5193fff640 1 -- 172.21.15.5:0/1786560798 >> 172.21.15.5:0/1786560798 conn(0x7f51940cec10 msgr2=0x7f51940aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.969+0000 7f5193fff640 1 -- 172.21.15.5:0/1786560798 shutdown_connections 2023-11-24T06:16:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:45.969+0000 7f5193fff640 1 -- 172.21.15.5:0/1786560798 wait complete. 2023-11-24T06:16:45.991 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.219+0000 7f2cbe6e0640 1 Processor -- start 2023-11-24T06:16:46.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.220+0000 7f2cbe6e0640 1 -- start start 2023-11-24T06:16:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.221+0000 7f2cbe6e0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2cb809cb30 0x7f2cb80abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.221+0000 7f2cbe6e0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb815b730 con 0x7f2cb815b350 2023-11-24T06:16:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.221+0000 7f2cbe6e0640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb80ac740 con 0x7f2cb80adb60 2023-11-24T06:16:46.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.221+0000 7f2cbe6e0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb80ac460 con 0x7f2cb809cb30 2023-11-24T06:16:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.222+0000 7f2cb7fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2cb815b350 0x7f2cb80b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56172/0 (socket says 172.21.15.5:56172) 2023-11-24T06:16:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.222+0000 7f2cb7fff640 1 -- 172.21.15.5:0/695137826 learned_addr learned my addr 172.21.15.5:0/695137826 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.222+0000 7f2caffff640 1 --2- 172.21.15.5:0/695137826 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2cb809cb30 0x7f2cb80abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.222+0000 7f2cb77fe640 1 -- 172.21.15.5:0/695137826 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839750525 0 0) 0x7f2ca8001110 con 0x7f2cb815b350 2023-11-24T06:16:46.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.222+0000 7f2cb77fe640 1 -- 172.21.15.5:0/695137826 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ca4003520 con 0x7f2cb815b350 2023-11-24T06:16:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.223+0000 7f2cb77fe640 1 -- 172.21.15.5:0/695137826 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 665335028 0 0) 0x7f2ca4003520 con 0x7f2cb815b350 2023-11-24T06:16:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.223+0000 7f2cb77fe640 1 -- 172.21.15.5:0/695137826 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2cb809cb30 msgr2=0x7f2cb80abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.223+0000 7f2cb77fe640 1 --2- 172.21.15.5:0/695137826 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2cb809cb30 0x7f2cb80abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.223+0000 7f2cb77fe640 1 -- 172.21.15.5:0/695137826 >> v1:172.21.15.6:6789/0 conn(0x7f2cb80adb60 legacy=0x7f2cb80adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:46.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.223+0000 7f2cb77fe640 1 -- 172.21.15.5:0/695137826 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb80b2a50 con 0x7f2cb815b350 2023-11-24T06:16:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.223+0000 7f2cb77fe640 1 -- 172.21.15.5:0/695137826 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f2ca8001800 con 0x7f2cb815b350 2023-11-24T06:16:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.223+0000 7f2cb77fe640 1 -- 172.21.15.5:0/695137826 >> v1:172.21.15.5:6789/0 conn(0x7f2cb815b350 legacy=0x7f2cb80b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.223+0000 7f2cb77fe640 1 --2- 172.21.15.5:0/695137826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca40039a0 0x7f2ca4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.223+0000 7f2cb77fe640 1 --2- 172.21.15.5:0/695137826 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2cb809cb30 0x7f2ca40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.223+0000 7f2cb77fe640 1 --2- 172.21.15.5:0/695137826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca4004d00 0x7f2ca4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.223+0000 7f2cb77fe640 1 -- 172.21.15.5:0/695137826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb80ac460 con 0x7f2ca40039a0 2023-11-24T06:16:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.223+0000 7f2cb77fe640 1 -- 172.21.15.5:0/695137826 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca400ba30 con 0x7f2ca4004d00 2023-11-24T06:16:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.223+0000 7f2cb77fe640 1 -- 172.21.15.5:0/695137826 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca400bbf0 con 0x7f2cb809cb30 2023-11-24T06:16:46.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.223+0000 7f2cb7fff640 1 --2- 172.21.15.5:0/695137826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca40039a0 0x7f2ca4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.224+0000 7f2caffff640 1 --2- 172.21.15.5:0/695137826 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2cb809cb30 0x7f2ca40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.224+0000 7f2cbcc7d640 1 --2- 172.21.15.5:0/695137826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca4004d00 0x7f2ca4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.224+0000 7f2cb7fff640 1 -- 172.21.15.5:0/695137826 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2cb809cb30 msgr2=0x7f2ca40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.224+0000 7f2cb7fff640 1 --2- 172.21.15.5:0/695137826 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2cb809cb30 0x7f2ca40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.224+0000 7f2cb7fff640 1 -- 172.21.15.5:0/695137826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca4004d00 msgr2=0x7f2ca4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.224+0000 7f2cb7fff640 1 --2- 172.21.15.5:0/695137826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca4004d00 0x7f2ca4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.224+0000 7f2cb7fff640 1 -- 172.21.15.5:0/695137826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca4003520 con 0x7f2ca40039a0 2023-11-24T06:16:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.224+0000 7f2cbcc7d640 1 --2- 172.21.15.5:0/695137826 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2ca4004d00 0x7f2ca4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:46.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.225+0000 7f2cb7fff640 1 --2- 172.21.15.5:0/695137826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca40039a0 0x7f2ca4003dc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800c210 tx=0x7f2ca40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2791aabffa7c4763 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.225+0000 7f2cb77fe640 1 -- 172.21.15.5:0/695137826 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8005ad0 con 0x7f2ca40039a0 2023-11-24T06:16:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.225+0000 7f2cb77fe640 1 -- 172.21.15.5:0/695137826 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca8005c50 con 0x7f2ca40039a0 2023-11-24T06:16:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.225+0000 7f2cb77fe640 1 -- 172.21.15.5:0/695137826 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca802c890 con 0x7f2ca40039a0 2023-11-24T06:16:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.226+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/695137826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca40039a0 msgr2=0x7f2ca4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.226+0000 7f2cbe6e0640 1 --2- 172.21.15.5:0/695137826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca40039a0 0x7f2ca4003dc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800c210 tx=0x7f2ca40057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.226+0000 7f2cbcc7d640 1 -- 172.21.15.5:0/695137826 reap_dead start 2023-11-24T06:16:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.226+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/695137826 shutdown_connections 2023-11-24T06:16:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.226+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/695137826 >> 172.21.15.5:0/695137826 conn(0x7f2cb80b0ba0 msgr2=0x7f2cb80b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.226+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/695137826 shutdown_connections 2023-11-24T06:16:46.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.226+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/695137826 wait complete. 2023-11-24T06:16:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.228+0000 7f2cbe6e0640 1 Processor -- start 2023-11-24T06:16:46.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.228+0000 7f2cbe6e0640 1 -- start start 2023-11-24T06:16:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.229+0000 7f2cbe6e0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ca40039a0 0x7f2cb813b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.229+0000 7f2cbe6e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb813b660 0x7f2cb8150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.229+0000 7f2cbe6e0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cb8151050 0x7f2cb81f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.229+0000 7f2cbe6e0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb813bbf0 con 0x7f2cb813b660 2023-11-24T06:16:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.229+0000 7f2cbe6e0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb8151550 con 0x7f2cb8151050 2023-11-24T06:16:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.229+0000 7f2cbe6e0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb8151830 con 0x7f2ca40039a0 2023-11-24T06:16:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.229+0000 7f2caffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb813b660 0x7f2cb8150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.229+0000 7f2caffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb813b660 0x7f2cb8150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40790/0 (socket says 172.21.15.5:40790) 2023-11-24T06:16:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.229+0000 7f2caffff640 1 -- 172.21.15.5:0/872330268 learned_addr learned my addr 172.21.15.5:0/872330268 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.230+0000 7f2cb7fff640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ca40039a0 0x7f2cb813b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.230+0000 7f2cbcc7d640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cb8151050 0x7f2cb81f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.230+0000 7f2caffff640 1 -- 172.21.15.5:0/872330268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ca40039a0 msgr2=0x7f2cb813b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.230+0000 7f2caffff640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ca40039a0 0x7f2cb813b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.230+0000 7f2caffff640 1 -- 172.21.15.5:0/872330268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cb8151050 msgr2=0x7f2cb81f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.230+0000 7f2caffff640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cb8151050 0x7f2cb81f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.230+0000 7f2caffff640 1 -- 172.21.15.5:0/872330268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb813be90 con 0x7f2cb813b660 2023-11-24T06:16:46.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.230+0000 7f2cbcc7d640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cb8151050 0x7f2cb81f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:46.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.230+0000 7f2caffff640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb813b660 0x7f2cb8150b10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0011600 tx=0x7f2ca0011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.231+0000 7f2cb57fa640 1 -- 172.21.15.5:0/872330268 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000f920 con 0x7f2cb813b660 2023-11-24T06:16:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.231+0000 7f2cb57fa640 1 -- 172.21.15.5:0/872330268 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca000fa80 con 0x7f2cb813b660 2023-11-24T06:16:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.231+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/872330268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb8151b10 con 0x7f2cb813b660 2023-11-24T06:16:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.231+0000 7f2cb57fa640 1 -- 172.21.15.5:0/872330268 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000fdf0 con 0x7f2cb813b660 2023-11-24T06:16:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.231+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/872330268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb814ef80 con 0x7f2cb813b660 2023-11-24T06:16:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.232+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/872330268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb809d200 con 0x7f2cb813b660 2023-11-24T06:16:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.233+0000 7f2cb57fa640 1 -- 172.21.15.5:0/872330268 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ca0015460 con 0x7f2cb813b660 2023-11-24T06:16:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.234+0000 7f2cb57fa640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2c9806ae40 0x7f2c9806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.234+0000 7f2cb57fa640 1 -- 172.21.15.5:0/872330268 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f2ca0089b30 con 0x7f2cb813b660 2023-11-24T06:16:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.234+0000 7f2cb7fff640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2c9806ae40 0x7f2c9806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.235+0000 7f2cb7fff640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2c9806ae40 0x7f2c9806d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8034000 tx=0x7f2ca8009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:46.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.239+0000 7f2cb57fa640 1 -- 172.21.15.5:0/872330268 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ca0054ab0 con 0x7f2cb813b660 2023-11-24T06:16:46.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.432+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/872330268 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2cb804e920 con 0x7f2cb813b660 2023-11-24T06:16:46.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.433+0000 7f2cb57fa640 1 -- 172.21.15.5:0/872330268 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2ca00220a0 con 0x7f2cb813b660 2023-11-24T06:16:46.436 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:46.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.436+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/872330268 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2c9806ae40 msgr2=0x7f2c9806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:46.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.436+0000 7f2cbe6e0640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2c9806ae40 0x7f2c9806d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2ca8034000 tx=0x7f2ca8009030 comp rx=0 tx=0).stop 2023-11-24T06:16:46.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.436+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/872330268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb813b660 msgr2=0x7f2cb8150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:46.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.436+0000 7f2cbe6e0640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb813b660 0x7f2cb8150b10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0011600 tx=0x7f2ca0011ac0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.436+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/872330268 shutdown_connections 2023-11-24T06:16:46.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.436+0000 7f2cbe6e0640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2c9806ae40 0x7f2c9806d300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.436+0000 7f2cbe6e0640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2cb8151050 0x7f2cb81f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.436+0000 7f2cbe6e0640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cb813b660 0x7f2cb8150b10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.436+0000 7f2cbe6e0640 1 --2- 172.21.15.5:0/872330268 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2ca40039a0 0x7f2cb813b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.436+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/872330268 >> 172.21.15.5:0/872330268 conn(0x7f2cb80b0ba0 msgr2=0x7f2cb806db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:46.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.437+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/872330268 shutdown_connections 2023-11-24T06:16:46.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.437+0000 7f2cbe6e0640 1 -- 172.21.15.5:0/872330268 wait complete. 2023-11-24T06:16:46.458 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:16:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.684+0000 7feb6b59e640 1 Processor -- start 2023-11-24T06:16:46.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.684+0000 7feb6b59e640 1 -- start start 2023-11-24T06:16:46.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.686+0000 7feb6b59e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb6c1416f0 0x7feb6c15ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:46.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.686+0000 7feb6b59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb6c06d230 0x7feb6c15f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:46.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.686+0000 7feb6b59e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb6c070bf0 con 0x7feb6c141e80 2023-11-24T06:16:46.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.686+0000 7feb6b59e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb6c06db30 con 0x7feb6c06d230 2023-11-24T06:16:46.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.686+0000 7feb6b59e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c06d850 con 0x7feb6c1416f0 2023-11-24T06:16:46.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.687+0000 7feb6ad9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb6c06d230 0x7feb6c15f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:46.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.687+0000 7feb6ad9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb6c06d230 0x7feb6c15f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40806/0 (socket says 172.21.15.5:40806) 2023-11-24T06:16:46.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.687+0000 7feb6ad9d640 1 -- 172.21.15.5:0/3599365015 learned_addr learned my addr 172.21.15.5:0/3599365015 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:46.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.687+0000 7feb6a59c640 1 --2- 172.21.15.5:0/3599365015 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb6c1416f0 0x7feb6c15ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:46.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.687+0000 7feb6ad9d640 1 -- 172.21.15.5:0/3599365015 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb6c1416f0 msgr2=0x7feb6c15ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:46.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.687+0000 7feb6ad9d640 1 --2- 172.21.15.5:0/3599365015 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb6c1416f0 0x7feb6c15ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.687+0000 7feb6ad9d640 1 -- 172.21.15.5:0/3599365015 >> v1:172.21.15.5:6789/0 conn(0x7feb6c141e80 legacy=0x7feb6c06cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:46.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.687+0000 7feb6ad9d640 1 -- 172.21.15.5:0/3599365015 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6c0705a0 con 0x7feb6c06d230 2023-11-24T06:16:46.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.687+0000 7feb6a59c640 1 --2- 172.21.15.5:0/3599365015 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb6c1416f0 0x7feb6c15ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:46.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.688+0000 7feb6ad9d640 1 --2- 172.21.15.5:0/3599365015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb6c06d230 0x7feb6c15f290 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feb64009e50 tx=0x7feb6402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c928e28991c13713 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:46.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.689+0000 7feb69d9b640 1 -- 172.21.15.5:0/3599365015 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6402cb00 con 0x7feb6c06d230 2023-11-24T06:16:46.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.689+0000 7feb69d9b640 1 -- 172.21.15.5:0/3599365015 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6402cc60 con 0x7feb6c06d230 2023-11-24T06:16:46.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.689+0000 7feb69d9b640 1 -- 172.21.15.5:0/3599365015 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb640343f0 con 0x7feb6c06d230 2023-11-24T06:16:46.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.689+0000 7feb6b59e640 1 -- 172.21.15.5:0/3599365015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb6c06d230 msgr2=0x7feb6c15f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:46.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.689+0000 7feb6b59e640 1 --2- 172.21.15.5:0/3599365015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb6c06d230 0x7feb6c15f290 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7feb64009e50 tx=0x7feb6402bf60 comp rx=0 tx=0).stop 2023-11-24T06:16:46.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.690+0000 7feb6b59e640 1 -- 172.21.15.5:0/3599365015 shutdown_connections 2023-11-24T06:16:46.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.690+0000 7feb6b59e640 1 --2- 172.21.15.5:0/3599365015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb6c06d230 0x7feb6c15f290 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.690+0000 7feb6b59e640 1 --2- 172.21.15.5:0/3599365015 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb6c1416f0 0x7feb6c15ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.690+0000 7feb6b59e640 1 -- 172.21.15.5:0/3599365015 >> 172.21.15.5:0/3599365015 conn(0x7feb6c0b0840 msgr2=0x7feb6c0b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:46.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.690+0000 7feb6b59e640 1 -- 172.21.15.5:0/3599365015 shutdown_connections 2023-11-24T06:16:46.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.690+0000 7feb6b59e640 1 -- 172.21.15.5:0/3599365015 wait complete. 2023-11-24T06:16:46.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.692+0000 7feb6b59e640 1 Processor -- start 2023-11-24T06:16:46.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.692+0000 7feb6b59e640 1 -- start start 2023-11-24T06:16:46.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.693+0000 7feb6b59e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb6c06d230 0x7feb6c1e4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:46.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.693+0000 7feb6b59e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb6c1416f0 0x7feb6c0b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:46.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.693+0000 7feb6b59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb6c141e80 0x7feb6c15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:46.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.693+0000 7feb6b59e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c1e8b90 con 0x7feb6c141e80 2023-11-24T06:16:46.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.693+0000 7feb6b59e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c1e7f80 con 0x7feb6c06d230 2023-11-24T06:16:46.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.693+0000 7feb6b59e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c1e8260 con 0x7feb6c1416f0 2023-11-24T06:16:46.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.693+0000 7feb6ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb6c141e80 0x7feb6c15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:46.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.693+0000 7feb6ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb6c141e80 0x7feb6c15a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40818/0 (socket says 172.21.15.5:40818) 2023-11-24T06:16:46.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.693+0000 7feb6ad9d640 1 -- 172.21.15.5:0/3269599211 learned_addr learned my addr 172.21.15.5:0/3269599211 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:46.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.694+0000 7feb6a59c640 1 --2- 172.21.15.5:0/3269599211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb6c06d230 0x7feb6c1e4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:46.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.694+0000 7feb63fff640 1 --2- 172.21.15.5:0/3269599211 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb6c1416f0 0x7feb6c0b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.694+0000 7feb6ad9d640 1 -- 172.21.15.5:0/3269599211 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb6c1416f0 msgr2=0x7feb6c0b32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.694+0000 7feb6ad9d640 1 --2- 172.21.15.5:0/3269599211 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb6c1416f0 0x7feb6c0b32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.694+0000 7feb6ad9d640 1 -- 172.21.15.5:0/3269599211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb6c06d230 msgr2=0x7feb6c1e4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.694+0000 7feb6ad9d640 1 --2- 172.21.15.5:0/3269599211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb6c06d230 0x7feb6c1e4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.694+0000 7feb6ad9d640 1 -- 172.21.15.5:0/3269599211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb64009b30 con 0x7feb6c141e80 2023-11-24T06:16:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.694+0000 7feb6ad9d640 1 --2- 172.21.15.5:0/3269599211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb6c141e80 0x7feb6c15a940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feb64009e50 tx=0x7feb64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.694+0000 7feb62ffd640 1 -- 172.21.15.5:0/3269599211 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb64009510 con 0x7feb6c141e80 2023-11-24T06:16:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.695+0000 7feb62ffd640 1 -- 172.21.15.5:0/3269599211 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb6403e450 con 0x7feb6c141e80 2023-11-24T06:16:46.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.695+0000 7feb62ffd640 1 -- 172.21.15.5:0/3269599211 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb6403e800 con 0x7feb6c141e80 2023-11-24T06:16:46.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.695+0000 7feb6b59e640 1 -- 172.21.15.5:0/3269599211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb6c1e8540 con 0x7feb6c141e80 2023-11-24T06:16:46.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.695+0000 7feb6b59e640 1 -- 172.21.15.5:0/3269599211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb6c1e4f60 con 0x7feb6c141e80 2023-11-24T06:16:46.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.696+0000 7feb6b59e640 1 -- 172.21.15.5:0/3269599211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb6c0cca10 con 0x7feb6c141e80 2023-11-24T06:16:46.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.697+0000 7feb62ffd640 1 -- 172.21.15.5:0/3269599211 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feb6403e960 con 0x7feb6c141e80 2023-11-24T06:16:46.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.698+0000 7feb62ffd640 1 --2- 172.21.15.5:0/3269599211 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7feb4c06ae40 0x7feb4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:46.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.698+0000 7feb62ffd640 1 -- 172.21.15.5:0/3269599211 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7feb640ac530 con 0x7feb6c141e80 2023-11-24T06:16:46.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.698+0000 7feb6a59c640 1 --2- 172.21.15.5:0/3269599211 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7feb4c06ae40 0x7feb4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:46.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.699+0000 7feb6a59c640 1 --2- 172.21.15.5:0/3269599211 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7feb4c06ae40 0x7feb4c06d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0b1a40 tx=0x7feb5c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:46.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.702+0000 7feb62ffd640 1 -- 172.21.15.5:0/3269599211 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feb64077600 con 0x7feb6c141e80 2023-11-24T06:16:46.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.910+0000 7feb6b59e640 1 -- 172.21.15.5:0/3269599211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feb6c04e840 con 0x7feb6c141e80 2023-11-24T06:16:46.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.911+0000 7feb62ffd640 1 -- 172.21.15.5:0/3269599211 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7feb6407ac20 con 0x7feb6c141e80 2023-11-24T06:16:46.914 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2023-11-24T06:16:46.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.914+0000 7feb6b59e640 1 -- 172.21.15.5:0/3269599211 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7feb4c06ae40 msgr2=0x7feb4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:46.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.914+0000 7feb6b59e640 1 --2- 172.21.15.5:0/3269599211 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7feb4c06ae40 0x7feb4c06d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7feb6c0b1a40 tx=0x7feb5c008040 comp rx=0 tx=0).stop 2023-11-24T06:16:46.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.914+0000 7feb6b59e640 1 -- 172.21.15.5:0/3269599211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb6c141e80 msgr2=0x7feb6c15a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.914+0000 7feb6b59e640 1 --2- 172.21.15.5:0/3269599211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb6c141e80 0x7feb6c15a940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feb64009e50 tx=0x7feb64004a40 comp rx=0 tx=0).stop 2023-11-24T06:16:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.914+0000 7feb6b59e640 1 -- 172.21.15.5:0/3269599211 shutdown_connections 2023-11-24T06:16:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.914+0000 7feb6b59e640 1 --2- 172.21.15.5:0/3269599211 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7feb4c06ae40 0x7feb4c06d300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.914+0000 7feb6b59e640 1 --2- 172.21.15.5:0/3269599211 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7feb6c1416f0 0x7feb6c0b32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.914+0000 7feb6b59e640 1 --2- 172.21.15.5:0/3269599211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb6c141e80 0x7feb6c15a940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.914+0000 7feb6b59e640 1 --2- 172.21.15.5:0/3269599211 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7feb6c06d230 0x7feb6c1e4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:46.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.914+0000 7feb6b59e640 1 -- 172.21.15.5:0/3269599211 >> 172.21.15.5:0/3269599211 conn(0x7feb6c0b0840 msgr2=0x7feb6c0ad6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:46.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.914+0000 7feb6b59e640 1 -- 172.21.15.5:0/3269599211 shutdown_connections 2023-11-24T06:16:46.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:46.914+0000 7feb6b59e640 1 -- 172.21.15.5:0/3269599211 wait complete. 2023-11-24T06:16:46.935 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:16:46.935 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:16:47.000 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.231+0000 7f7c7ebf1640 1 Processor -- start 2023-11-24T06:16:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.231+0000 7f7c7ebf1640 1 -- start start 2023-11-24T06:16:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.233+0000 7f7c7ebf1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c780719c0 0x7f7c780ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.233+0000 7f7c7ebf1640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c780ad510 con 0x7f7c7806d520 2023-11-24T06:16:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.233+0000 7f7c7ebf1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c780acc50 con 0x7f7c78161930 2023-11-24T06:16:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.233+0000 7f7c7ebf1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c78071e60 con 0x7f7c780719c0 2023-11-24T06:16:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.233+0000 7f7c7d18e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7c7806d520 0x7f7c780b2080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56194/0 (socket says 172.21.15.5:56194) 2023-11-24T06:16:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.233+0000 7f7c7d18e640 1 -- 172.21.15.5:0/3237694727 learned_addr learned my addr 172.21.15.5:0/3237694727 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.234+0000 7f7c77fff640 1 --2- 172.21.15.5:0/3237694727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c780719c0 0x7f7c780ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.234+0000 7f7c777fe640 1 -- 172.21.15.5:0/3237694727 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874236357 0 0) 0x7f7c780ad510 con 0x7f7c7806d520 2023-11-24T06:16:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.234+0000 7f7c777fe640 1 -- 172.21.15.5:0/3237694727 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c68003520 con 0x7f7c7806d520 2023-11-24T06:16:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.234+0000 7f7c777fe640 1 -- 172.21.15.5:0/3237694727 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2332117418 0 0) 0x7f7c68003520 con 0x7f7c7806d520 2023-11-24T06:16:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.234+0000 7f7c777fe640 1 -- 172.21.15.5:0/3237694727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c780719c0 msgr2=0x7f7c780ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.234+0000 7f7c777fe640 1 --2- 172.21.15.5:0/3237694727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c780719c0 0x7f7c780ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.234+0000 7f7c777fe640 1 -- 172.21.15.5:0/3237694727 >> v1:172.21.15.6:6789/0 conn(0x7f7c78161930 legacy=0x7f7c780ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.234+0000 7f7c777fe640 1 -- 172.21.15.5:0/3237694727 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c780acec0 con 0x7f7c7806d520 2023-11-24T06:16:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.234+0000 7f7c777fe640 1 -- 172.21.15.5:0/3237694727 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f7c700019f0 con 0x7f7c7806d520 2023-11-24T06:16:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.235+0000 7f7c777fe640 1 -- 172.21.15.5:0/3237694727 >> v1:172.21.15.5:6789/0 conn(0x7f7c7806d520 legacy=0x7f7c780b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.235+0000 7f7c777fe640 1 --2- 172.21.15.5:0/3237694727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c680039a0 0x7f7c68003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.235+0000 7f7c777fe640 1 --2- 172.21.15.5:0/3237694727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c680047c0 0x7f7c68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.235+0000 7f7c777fe640 1 --2- 172.21.15.5:0/3237694727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c780719c0 0x7f7c68005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.235+0000 7f7c777fe640 1 -- 172.21.15.5:0/3237694727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c78071e60 con 0x7f7c680047c0 2023-11-24T06:16:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.235+0000 7f7c777fe640 1 -- 172.21.15.5:0/3237694727 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6800ba60 con 0x7f7c680039a0 2023-11-24T06:16:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.235+0000 7f7c777fe640 1 -- 172.21.15.5:0/3237694727 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6800bc20 con 0x7f7c780719c0 2023-11-24T06:16:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.235+0000 7f7c77fff640 1 --2- 172.21.15.5:0/3237694727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c680047c0 0x7f7c68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.235+0000 7f7c7d18e640 1 --2- 172.21.15.5:0/3237694727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c780719c0 0x7f7c68005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.235+0000 7f7c77fff640 1 -- 172.21.15.5:0/3237694727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c780719c0 msgr2=0x7f7c68005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.235+0000 7f7c7c98d640 1 --2- 172.21.15.5:0/3237694727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c680039a0 0x7f7c68003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.235+0000 7f7c77fff640 1 --2- 172.21.15.5:0/3237694727 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c780719c0 0x7f7c68005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.235+0000 7f7c77fff640 1 -- 172.21.15.5:0/3237694727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c680039a0 msgr2=0x7f7c68003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.235+0000 7f7c77fff640 1 --2- 172.21.15.5:0/3237694727 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c680039a0 0x7f7c68003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.236+0000 7f7c77fff640 1 -- 172.21.15.5:0/3237694727 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c68003520 con 0x7f7c680047c0 2023-11-24T06:16:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.237+0000 7f7c77fff640 1 --2- 172.21.15.5:0/3237694727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c680047c0 0x7f7c68004c40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009a90 tx=0x7f7c680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d66a24accb768d6f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.237+0000 7f7c777fe640 1 -- 172.21.15.5:0/3237694727 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6402c680 con 0x7f7c680047c0 2023-11-24T06:16:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.237+0000 7f7c777fe640 1 -- 172.21.15.5:0/3237694727 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6402c7e0 con 0x7f7c680047c0 2023-11-24T06:16:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.237+0000 7f7c777fe640 1 -- 172.21.15.5:0/3237694727 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6402cb70 con 0x7f7c680047c0 2023-11-24T06:16:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.238+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/3237694727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c680047c0 msgr2=0x7f7c68004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.238+0000 7f7c7ebf1640 1 --2- 172.21.15.5:0/3237694727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c680047c0 0x7f7c68004c40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009a90 tx=0x7f7c680057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.238+0000 7f7c7d18e640 1 -- 172.21.15.5:0/3237694727 reap_dead start 2023-11-24T06:16:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.238+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/3237694727 shutdown_connections 2023-11-24T06:16:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.238+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/3237694727 >> 172.21.15.5:0/3237694727 conn(0x7f7c780b0d80 msgr2=0x7f7c780b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.238+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/3237694727 shutdown_connections 2023-11-24T06:16:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.238+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/3237694727 wait complete. 2023-11-24T06:16:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.240+0000 7f7c7ebf1640 1 Processor -- start 2023-11-24T06:16:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.240+0000 7f7c7ebf1640 1 -- start start 2023-11-24T06:16:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.241+0000 7f7c7ebf1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c680047c0 0x7f7c780ce300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.241+0000 7f7c7ebf1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c78160980 0x7f7c78160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.241+0000 7f7c7ebf1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c780aee40 0x7f7c780af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.241+0000 7f7c7ebf1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c780af800 con 0x7f7c680047c0 2023-11-24T06:16:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.241+0000 7f7c7ebf1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c780ce840 con 0x7f7c78160980 2023-11-24T06:16:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.241+0000 7f7c7ebf1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c780cd260 con 0x7f7c780aee40 2023-11-24T06:16:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.241+0000 7f7c7c98d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c680047c0 0x7f7c780ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.241+0000 7f7c7c98d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c680047c0 0x7f7c780ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40844/0 (socket says 172.21.15.5:40844) 2023-11-24T06:16:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.241+0000 7f7c7c98d640 1 -- 172.21.15.5:0/2356174359 learned_addr learned my addr 172.21.15.5:0/2356174359 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.242+0000 7f7c7c98d640 1 -- 172.21.15.5:0/2356174359 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c780aee40 msgr2=0x7f7c780af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.242+0000 7f7c7c98d640 1 --2- 172.21.15.5:0/2356174359 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c780aee40 0x7f7c780af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.242+0000 7f7c7c98d640 1 -- 172.21.15.5:0/2356174359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c78160980 msgr2=0x7f7c78160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.242+0000 7f7c7c98d640 1 --2- 172.21.15.5:0/2356174359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c78160980 0x7f7c78160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.242+0000 7f7c7c98d640 1 -- 172.21.15.5:0/2356174359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c780cd540 con 0x7f7c680047c0 2023-11-24T06:16:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.242+0000 7f7c7c98d640 1 --2- 172.21.15.5:0/2356174359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c680047c0 0x7f7c780ce300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00d570 tx=0x7f7c6c00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.242+0000 7f7c757fa640 1 -- 172.21.15.5:0/2356174359 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c005480 con 0x7f7c680047c0 2023-11-24T06:16:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.242+0000 7f7c757fa640 1 -- 172.21.15.5:0/2356174359 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6c003610 con 0x7f7c680047c0 2023-11-24T06:16:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.242+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/2356174359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c780cd820 con 0x7f7c680047c0 2023-11-24T06:16:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.242+0000 7f7c757fa640 1 -- 172.21.15.5:0/2356174359 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6c003980 con 0x7f7c680047c0 2023-11-24T06:16:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.242+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/2356174359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c780cdde0 con 0x7f7c680047c0 2023-11-24T06:16:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.244+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/2356174359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c780cdf70 con 0x7f7c680047c0 2023-11-24T06:16:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.245+0000 7f7c757fa640 1 -- 172.21.15.5:0/2356174359 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c6c003ae0 con 0x7f7c680047c0 2023-11-24T06:16:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.245+0000 7f7c757fa640 1 --2- 172.21.15.5:0/2356174359 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7c5c06ae70 0x7f7c5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.246+0000 7f7c757fa640 1 -- 172.21.15.5:0/2356174359 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7c6c089730 con 0x7f7c680047c0 2023-11-24T06:16:47.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.246+0000 7f7c77fff640 1 --2- 172.21.15.5:0/2356174359 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7c5c06ae70 0x7f7c5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:47.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.246+0000 7f7c77fff640 1 --2- 172.21.15.5:0/2356174359 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7c5c06ae70 0x7f7c5c06d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7c64036000 tx=0x7f7c6402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:47.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.250+0000 7f7c757fa640 1 -- 172.21.15.5:0/2356174359 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c6c054620 con 0x7f7c680047c0 2023-11-24T06:16:47.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.444+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/2356174359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c780ae450 con 0x7f7c680047c0 2023-11-24T06:16:47.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.446+0000 7f7c757fa640 1 -- 172.21.15.5:0/2356174359 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16658 (secure 0 0 0) 0x7f7c6c001230 con 0x7f7c680047c0 2023-11-24T06:16:47.449 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:47.449 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:16:33.958286+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:47.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.448+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/2356174359 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7c5c06ae70 msgr2=0x7f7c5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:47.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.448+0000 7f7c7ebf1640 1 --2- 172.21.15.5:0/2356174359 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7c5c06ae70 0x7f7c5c06d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7c64036000 tx=0x7f7c6402e000 comp rx=0 tx=0).stop 2023-11-24T06:16:47.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.448+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/2356174359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c680047c0 msgr2=0x7f7c780ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:47.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.448+0000 7f7c7ebf1640 1 --2- 172.21.15.5:0/2356174359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c680047c0 0x7f7c780ce300 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c00d570 tx=0x7f7c6c00da30 comp rx=0 tx=0).stop 2023-11-24T06:16:47.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.449+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/2356174359 shutdown_connections 2023-11-24T06:16:47.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.449+0000 7f7c7ebf1640 1 --2- 172.21.15.5:0/2356174359 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7c5c06ae70 0x7f7c5c06d330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.449+0000 7f7c7ebf1640 1 --2- 172.21.15.5:0/2356174359 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7c780aee40 0x7f7c780af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.449+0000 7f7c7ebf1640 1 --2- 172.21.15.5:0/2356174359 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7c78160980 0x7f7c78160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.449+0000 7f7c7ebf1640 1 --2- 172.21.15.5:0/2356174359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c680047c0 0x7f7c780ce300 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.449+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/2356174359 >> 172.21.15.5:0/2356174359 conn(0x7f7c780b0d80 msgr2=0x7f7c78165380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:47.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.449+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/2356174359 shutdown_connections 2023-11-24T06:16:47.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.449+0000 7f7c7ebf1640 1 -- 172.21.15.5:0/2356174359 wait complete. 2023-11-24T06:16:47.472 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:47.473 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:47.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.701+0000 7fcf2de00640 1 Processor -- start 2023-11-24T06:16:47.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.702+0000 7fcf2de00640 1 -- start start 2023-11-24T06:16:47.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.704+0000 7fcf2de00640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fcf280ae270 0x7fcf280abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.704+0000 7fcf2de00640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2815b730 con 0x7fcf2815b350 2023-11-24T06:16:47.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.704+0000 7fcf2de00640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf280ac740 con 0x7fcf2809cb30 2023-11-24T06:16:47.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.704+0000 7fcf2de00640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf280ac460 con 0x7fcf280ae270 2023-11-24T06:16:47.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.704+0000 7fcf277fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcf2815b350 0x7fcf280b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56204/0 (socket says 172.21.15.5:56204) 2023-11-24T06:16:47.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.704+0000 7fcf277fe640 1 -- 172.21.15.5:0/3199000376 learned_addr learned my addr 172.21.15.5:0/3199000376 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:47.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.705+0000 7fcf27fff640 1 --2- 172.21.15.5:0/3199000376 >> v2:172.21.15.6:3300/0 conn(0x7fcf280ae270 0x7fcf280abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:47.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.705+0000 7fcf26ffd640 1 -- 172.21.15.5:0/3199000376 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2336654918 0 0) 0x7fcf18001110 con 0x7fcf2815b350 2023-11-24T06:16:47.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.705+0000 7fcf26ffd640 1 -- 172.21.15.5:0/3199000376 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf14003520 con 0x7fcf2815b350 2023-11-24T06:16:47.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.705+0000 7fcf26ffd640 1 -- 172.21.15.5:0/3199000376 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 994991837 0 0) 0x7fcf14003520 con 0x7fcf2815b350 2023-11-24T06:16:47.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.705+0000 7fcf26ffd640 1 -- 172.21.15.5:0/3199000376 >> v2:172.21.15.6:3300/0 conn(0x7fcf280ae270 msgr2=0x7fcf280abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:47.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.705+0000 7fcf26ffd640 1 --2- 172.21.15.5:0/3199000376 >> v2:172.21.15.6:3300/0 conn(0x7fcf280ae270 0x7fcf280abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.705+0000 7fcf26ffd640 1 -- 172.21.15.5:0/3199000376 >> v1:172.21.15.6:6789/0 conn(0x7fcf2809cb30 legacy=0x7fcf280adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:47.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.705+0000 7fcf26ffd640 1 -- 172.21.15.5:0/3199000376 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf280b2a50 con 0x7fcf2815b350 2023-11-24T06:16:47.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.705+0000 7fcf26ffd640 1 -- 172.21.15.5:0/3199000376 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fcf18001800 con 0x7fcf2815b350 2023-11-24T06:16:47.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.705+0000 7fcf26ffd640 1 -- 172.21.15.5:0/3199000376 >> v1:172.21.15.5:6789/0 conn(0x7fcf2815b350 legacy=0x7fcf280b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:47.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.705+0000 7fcf26ffd640 1 --2- 172.21.15.5:0/3199000376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf140039c0 0x7fcf14003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.705+0000 7fcf26ffd640 1 --2- 172.21.15.5:0/3199000376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf140047c0 0x7fcf14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.706+0000 7fcf26ffd640 1 --2- 172.21.15.5:0/3199000376 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf14005180 0x7fcf1400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.706+0000 7fcf26ffd640 1 -- 172.21.15.5:0/3199000376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf280ac460 con 0x7fcf140039c0 2023-11-24T06:16:47.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.706+0000 7fcf26ffd640 1 -- 172.21.15.5:0/3199000376 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1400c080 con 0x7fcf140047c0 2023-11-24T06:16:47.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.706+0000 7fcf26ffd640 1 -- 172.21.15.5:0/3199000376 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1400c240 con 0x7fcf14005180 2023-11-24T06:16:47.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.706+0000 7fcf277fe640 1 --2- 172.21.15.5:0/3199000376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf140039c0 0x7fcf14003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:47.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.706+0000 7fcf1ffff640 1 --2- 172.21.15.5:0/3199000376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf140047c0 0x7fcf14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:47.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.706+0000 7fcf277fe640 1 -- 172.21.15.5:0/3199000376 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf14005180 msgr2=0x7fcf1400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:47.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.706+0000 7fcf277fe640 1 --2- 172.21.15.5:0/3199000376 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf14005180 0x7fcf1400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.706+0000 7fcf277fe640 1 -- 172.21.15.5:0/3199000376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf140047c0 msgr2=0x7fcf14004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:47.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.706+0000 7fcf277fe640 1 --2- 172.21.15.5:0/3199000376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf140047c0 0x7fcf14004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.706+0000 7fcf277fe640 1 -- 172.21.15.5:0/3199000376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf14003520 con 0x7fcf140039c0 2023-11-24T06:16:47.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.706+0000 7fcf27fff640 1 -- 172.21.15.5:0/3199000376 reap_dead start 2023-11-24T06:16:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.707+0000 7fcf277fe640 1 --2- 172.21.15.5:0/3199000376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf140039c0 0x7fcf14003de0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcf18007fc0 tx=0x7fcf140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537868e7be7e6748 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.708+0000 7fcf26ffd640 1 -- 172.21.15.5:0/3199000376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18005ad0 con 0x7fcf140039c0 2023-11-24T06:16:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.708+0000 7fcf26ffd640 1 -- 172.21.15.5:0/3199000376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf18005c50 con 0x7fcf140039c0 2023-11-24T06:16:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.708+0000 7fcf26ffd640 1 -- 172.21.15.5:0/3199000376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1802c720 con 0x7fcf140039c0 2023-11-24T06:16:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.708+0000 7fcf2de00640 1 -- 172.21.15.5:0/3199000376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf140039c0 msgr2=0x7fcf14003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.708+0000 7fcf2de00640 1 --2- 172.21.15.5:0/3199000376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf140039c0 0x7fcf14003de0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcf18007fc0 tx=0x7fcf140057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.708+0000 7fcf2de00640 1 -- 172.21.15.5:0/3199000376 shutdown_connections 2023-11-24T06:16:47.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.708+0000 7fcf2de00640 1 --2- 172.21.15.5:0/3199000376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf140039c0 0x7fcf14003de0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.708+0000 7fcf2de00640 1 -- 172.21.15.5:0/3199000376 >> 172.21.15.5:0/3199000376 conn(0x7fcf280b0ba0 msgr2=0x7fcf280b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:47.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.708+0000 7fcf2de00640 1 -- 172.21.15.5:0/3199000376 shutdown_connections 2023-11-24T06:16:47.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.709+0000 7fcf2de00640 1 -- 172.21.15.5:0/3199000376 wait complete. 2023-11-24T06:16:47.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.710+0000 7fcf2de00640 1 Processor -- start 2023-11-24T06:16:47.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.711+0000 7fcf2de00640 1 -- start start 2023-11-24T06:16:47.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.711+0000 7fcf2de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf140039c0 0x7fcf2814a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.711+0000 7fcf2de00640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf2815e050 0x7fcf2814c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf2de00640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf28070490 0x7fcf28157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf2de00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2814a6e0 con 0x7fcf140039c0 2023-11-24T06:16:47.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf2de00640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28070e00 con 0x7fcf2815e050 2023-11-24T06:16:47.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf2de00640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf280710e0 con 0x7fcf28070490 2023-11-24T06:16:47.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf140039c0 0x7fcf2814a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:47.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf140039c0 0x7fcf2814a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40862/0 (socket says 172.21.15.5:40862) 2023-11-24T06:16:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf277fe640 1 -- 172.21.15.5:0/3837749024 learned_addr learned my addr 172.21.15.5:0/3837749024 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:47.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf27fff640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf28070490 0x7fcf28157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:47.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf1ffff640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf2815e050 0x7fcf2814c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:47.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf277fe640 1 -- 172.21.15.5:0/3837749024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf28070490 msgr2=0x7fcf28157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:47.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf277fe640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf28070490 0x7fcf28157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf277fe640 1 -- 172.21.15.5:0/3837749024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf2815e050 msgr2=0x7fcf2814c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:47.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf277fe640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf2815e050 0x7fcf2814c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf277fe640 1 -- 172.21.15.5:0/3837749024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf280713c0 con 0x7fcf140039c0 2023-11-24T06:16:47.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.712+0000 7fcf1ffff640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf2815e050 0x7fcf2814c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:47.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.713+0000 7fcf277fe640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf140039c0 0x7fcf2814a1a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcf1802c390 tx=0x7fcf1802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:47.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.713+0000 7fcf24ff9640 1 -- 172.21.15.5:0/3837749024 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1803c040 con 0x7fcf140039c0 2023-11-24T06:16:47.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.713+0000 7fcf24ff9640 1 -- 172.21.15.5:0/3837749024 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1802cb30 con 0x7fcf140039c0 2023-11-24T06:16:47.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.713+0000 7fcf24ff9640 1 -- 172.21.15.5:0/3837749024 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18000ed0 con 0x7fcf140039c0 2023-11-24T06:16:47.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.713+0000 7fcf2de00640 1 -- 172.21.15.5:0/3837749024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2815b8d0 con 0x7fcf140039c0 2023-11-24T06:16:47.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.713+0000 7fcf2de00640 1 -- 172.21.15.5:0/3837749024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2814c560 con 0x7fcf140039c0 2023-11-24T06:16:47.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.715+0000 7fcf2de00640 1 -- 172.21.15.5:0/3837749024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2809d200 con 0x7fcf140039c0 2023-11-24T06:16:47.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.716+0000 7fcf24ff9640 1 -- 172.21.15.5:0/3837749024 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcf18032070 con 0x7fcf140039c0 2023-11-24T06:16:47.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.716+0000 7fcf24ff9640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fcf0806ae70 0x7fcf0806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:47.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.717+0000 7fcf24ff9640 1 -- 172.21.15.5:0/3837749024 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fcf180aa7c0 con 0x7fcf140039c0 2023-11-24T06:16:47.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.717+0000 7fcf1ffff640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fcf0806ae70 0x7fcf0806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:47.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.718+0000 7fcf1ffff640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fcf0806ae70 0x7fcf0806d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcf28157ea0 tx=0x7fcf10009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:47.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.721+0000 7fcf24ff9640 1 -- 172.21.15.5:0/3837749024 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf18075900 con 0x7fcf140039c0 2023-11-24T06:16:47.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.918+0000 7fcf2de00640 1 -- 172.21.15.5:0/3837749024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf2815b730 con 0x7fcf140039c0 2023-11-24T06:16:47.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.919+0000 7fcf24ff9640 1 -- 172.21.15.5:0/3837749024 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf2815b730 con 0x7fcf140039c0 2023-11-24T06:16:47.922 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:47.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.921+0000 7fcf2de00640 1 -- 172.21.15.5:0/3837749024 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fcf0806ae70 msgr2=0x7fcf0806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:47.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.921+0000 7fcf2de00640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fcf0806ae70 0x7fcf0806d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcf28157ea0 tx=0x7fcf10009040 comp rx=0 tx=0).stop 2023-11-24T06:16:47.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.921+0000 7fcf2de00640 1 -- 172.21.15.5:0/3837749024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf140039c0 msgr2=0x7fcf2814a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:47.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.921+0000 7fcf2de00640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf140039c0 0x7fcf2814a1a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcf1802c390 tx=0x7fcf1802e070 comp rx=0 tx=0).stop 2023-11-24T06:16:47.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.921+0000 7fcf2de00640 1 -- 172.21.15.5:0/3837749024 shutdown_connections 2023-11-24T06:16:47.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.921+0000 7fcf2de00640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fcf0806ae70 0x7fcf0806d330 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.921+0000 7fcf2de00640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fcf28070490 0x7fcf28157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.921+0000 7fcf2de00640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fcf2815e050 0x7fcf2814c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.921+0000 7fcf2de00640 1 --2- 172.21.15.5:0/3837749024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf140039c0 0x7fcf2814a1a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:47.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.921+0000 7fcf2de00640 1 -- 172.21.15.5:0/3837749024 >> 172.21.15.5:0/3837749024 conn(0x7fcf280b0ba0 msgr2=0x7fcf280adb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:47.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.921+0000 7fcf2de00640 1 -- 172.21.15.5:0/3837749024 shutdown_connections 2023-11-24T06:16:47.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:47.921+0000 7fcf2de00640 1 -- 172.21.15.5:0/3837749024 wait complete. 2023-11-24T06:16:47.945 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.174+0000 7f03b916c640 1 Processor -- start 2023-11-24T06:16:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.174+0000 7f03b916c640 1 -- start start 2023-11-24T06:16:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.176+0000 7f03b916c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f03b4153ea0 0x7f03b40ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.176+0000 7f03b916c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03b4070850 0x7f03b4071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:48.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.176+0000 7f03b916c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b40ce650 con 0x7f03b4154280 2023-11-24T06:16:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.176+0000 7f03b916c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b4154710 con 0x7f03b4070850 2023-11-24T06:16:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.176+0000 7f03b916c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b415e0a0 con 0x7f03b4153ea0 2023-11-24T06:16:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.176+0000 7f03b896b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03b4070850 0x7f03b4071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.176+0000 7f03b896b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f03b4070850 0x7f03b4071350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40870/0 (socket says 172.21.15.5:40870) 2023-11-24T06:16:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.176+0000 7f03b896b640 1 -- 172.21.15.5:0/1637131566 learned_addr learned my addr 172.21.15.5:0/1637131566 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.176+0000 7f03b3fff640 1 --2- 172.21.15.5:0/1637131566 >> v2:172.21.15.6:3300/0 conn(0x7f03b4153ea0 0x7f03b40ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.177+0000 7f03b896b640 1 -- 172.21.15.5:0/1637131566 >> v2:172.21.15.6:3300/0 conn(0x7f03b4153ea0 msgr2=0x7f03b40ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.177+0000 7f03b896b640 1 --2- 172.21.15.5:0/1637131566 >> v2:172.21.15.6:3300/0 conn(0x7f03b4153ea0 0x7f03b40ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.177+0000 7f03b896b640 1 -- 172.21.15.5:0/1637131566 >> v1:172.21.15.6:6789/0 conn(0x7f03b4154280 legacy=0x7f03b415d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.177+0000 7f03b896b640 1 -- 172.21.15.5:0/1637131566 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b415e380 con 0x7f03b4070850 2023-11-24T06:16:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.178+0000 7f03b896b640 1 --2- 172.21.15.5:0/1637131566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03b4070850 0x7f03b4071350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f03b40cfc10 tx=0x7f03ac02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eaec69fce2bb890 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:48.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.178+0000 7f03b2ffd640 1 -- 172.21.15.5:0/1637131566 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac02ca80 con 0x7f03b4070850 2023-11-24T06:16:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.178+0000 7f03b2ffd640 1 -- 172.21.15.5:0/1637131566 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac02cbe0 con 0x7f03b4070850 2023-11-24T06:16:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.178+0000 7f03b2ffd640 1 -- 172.21.15.5:0/1637131566 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac0343f0 con 0x7f03b4070850 2023-11-24T06:16:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.178+0000 7f03b916c640 1 -- 172.21.15.5:0/1637131566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03b4070850 msgr2=0x7f03b4071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.178+0000 7f03b916c640 1 --2- 172.21.15.5:0/1637131566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03b4070850 0x7f03b4071350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f03b40cfc10 tx=0x7f03ac02bee0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.179+0000 7f03b916c640 1 -- 172.21.15.5:0/1637131566 shutdown_connections 2023-11-24T06:16:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.179+0000 7f03b916c640 1 --2- 172.21.15.5:0/1637131566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03b4070850 0x7f03b4071350 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.179+0000 7f03b916c640 1 --2- 172.21.15.5:0/1637131566 >> v2:172.21.15.6:3300/0 conn(0x7f03b4153ea0 0x7f03b40ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.179+0000 7f03b916c640 1 -- 172.21.15.5:0/1637131566 >> 172.21.15.5:0/1637131566 conn(0x7f03b40cec10 msgr2=0x7f03b40cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.179+0000 7f03b916c640 1 -- 172.21.15.5:0/1637131566 shutdown_connections 2023-11-24T06:16:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.179+0000 7f03b916c640 1 -- 172.21.15.5:0/1637131566 wait complete. 2023-11-24T06:16:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.181+0000 7f03b916c640 1 Processor -- start 2023-11-24T06:16:48.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.181+0000 7f03b916c640 1 -- start start 2023-11-24T06:16:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.182+0000 7f03b916c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03b4070850 0x7f03b4144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.182+0000 7f03b916c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03b4153ea0 0x7f03b414ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.182+0000 7f03b916c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03b4154280 0x7f03b414fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.182+0000 7f03b916c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b4154fb0 con 0x7f03b4154280 2023-11-24T06:16:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.182+0000 7f03b916c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b4144fe0 con 0x7f03b4070850 2023-11-24T06:16:48.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.182+0000 7f03b916c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b414fa60 con 0x7f03b4153ea0 2023-11-24T06:16:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.182+0000 7f03b896b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03b4154280 0x7f03b414fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.182+0000 7f03b896b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03b4154280 0x7f03b414fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40882/0 (socket says 172.21.15.5:40882) 2023-11-24T06:16:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.182+0000 7f03b896b640 1 -- 172.21.15.5:0/4014629142 learned_addr learned my addr 172.21.15.5:0/4014629142 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.182+0000 7f03b3fff640 1 --2- 172.21.15.5:0/4014629142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03b4070850 0x7f03b4144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.183+0000 7f03b896b640 1 -- 172.21.15.5:0/4014629142 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03b4153ea0 msgr2=0x7f03b414ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.183+0000 7f03b896b640 1 --2- 172.21.15.5:0/4014629142 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03b4153ea0 0x7f03b414ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.183+0000 7f03b896b640 1 -- 172.21.15.5:0/4014629142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03b4070850 msgr2=0x7f03b4144aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.183+0000 7f03b896b640 1 --2- 172.21.15.5:0/4014629142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03b4070850 0x7f03b4144aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.183+0000 7f03b896b640 1 -- 172.21.15.5:0/4014629142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac009b30 con 0x7f03b4154280 2023-11-24T06:16:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.183+0000 7f03b896b640 1 --2- 172.21.15.5:0/4014629142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03b4154280 0x7f03b414fdf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009f40 tx=0x7f03ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.185+0000 7f03b0ff9640 1 -- 172.21.15.5:0/4014629142 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac009510 con 0x7f03b4154280 2023-11-24T06:16:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.185+0000 7f03b0ff9640 1 -- 172.21.15.5:0/4014629142 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac03e450 con 0x7f03b4154280 2023-11-24T06:16:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.185+0000 7f03b916c640 1 -- 172.21.15.5:0/4014629142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b414f780 con 0x7f03b4154280 2023-11-24T06:16:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.185+0000 7f03b0ff9640 1 -- 172.21.15.5:0/4014629142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac03e7e0 con 0x7f03b4154280 2023-11-24T06:16:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.185+0000 7f03b916c640 1 -- 172.21.15.5:0/4014629142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b4154bd0 con 0x7f03b4154280 2023-11-24T06:16:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.187+0000 7f03b916c640 1 -- 172.21.15.5:0/4014629142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b41550f0 con 0x7f03b4154280 2023-11-24T06:16:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.187+0000 7f03b0ff9640 1 -- 172.21.15.5:0/4014629142 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f03ac02cd50 con 0x7f03b4154280 2023-11-24T06:16:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.188+0000 7f03b0ff9640 1 --2- 172.21.15.5:0/4014629142 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f03a006ae40 0x7f03a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.188+0000 7f03b0ff9640 1 -- 172.21.15.5:0/4014629142 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f03ac0ac2c0 con 0x7f03b4154280 2023-11-24T06:16:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.188+0000 7f03b3fff640 1 --2- 172.21.15.5:0/4014629142 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f03a006ae40 0x7f03a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.189+0000 7f03b3fff640 1 --2- 172.21.15.5:0/4014629142 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f03a006ae40 0x7f03a006d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f03a4006fd0 tx=0x7f03a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.192+0000 7f03b0ff9640 1 -- 172.21.15.5:0/4014629142 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03b41550f0 con 0x7f03b4154280 2023-11-24T06:16:48.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.382+0000 7f03b916c640 1 -- 172.21.15.5:0/4014629142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03b41550f0 con 0x7f03b4154280 2023-11-24T06:16:48.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.383+0000 7f03b0ff9640 1 -- 172.21.15.5:0/4014629142 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03b41550f0 con 0x7f03b4154280 2023-11-24T06:16:48.386 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:48.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.386+0000 7f03b916c640 1 -- 172.21.15.5:0/4014629142 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f03a006ae40 msgr2=0x7f03a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:48.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.386+0000 7f03b916c640 1 --2- 172.21.15.5:0/4014629142 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f03a006ae40 0x7f03a006d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f03a4006fd0 tx=0x7f03a4008040 comp rx=0 tx=0).stop 2023-11-24T06:16:48.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.386+0000 7f03b916c640 1 -- 172.21.15.5:0/4014629142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03b4154280 msgr2=0x7f03b414fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:48.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.386+0000 7f03b916c640 1 --2- 172.21.15.5:0/4014629142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03b4154280 0x7f03b414fdf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009f40 tx=0x7f03ac004a40 comp rx=0 tx=0).stop 2023-11-24T06:16:48.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.386+0000 7f03b916c640 1 -- 172.21.15.5:0/4014629142 shutdown_connections 2023-11-24T06:16:48.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.386+0000 7f03b916c640 1 --2- 172.21.15.5:0/4014629142 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f03a006ae40 0x7f03a006d300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.386+0000 7f03b916c640 1 --2- 172.21.15.5:0/4014629142 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f03b4153ea0 0x7f03b414ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.386+0000 7f03b916c640 1 --2- 172.21.15.5:0/4014629142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03b4154280 0x7f03b414fdf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.386+0000 7f03b916c640 1 --2- 172.21.15.5:0/4014629142 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f03b4070850 0x7f03b4144aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.386+0000 7f03b916c640 1 -- 172.21.15.5:0/4014629142 >> 172.21.15.5:0/4014629142 conn(0x7f03b40cec10 msgr2=0x7f03b40ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:48.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.386+0000 7f03b916c640 1 -- 172.21.15.5:0/4014629142 shutdown_connections 2023-11-24T06:16:48.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.386+0000 7f03b916c640 1 -- 172.21.15.5:0/4014629142 wait complete. 2023-11-24T06:16:48.406 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.620+0000 7f1724a6b640 1 Processor -- start 2023-11-24T06:16:48.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.620+0000 7f1724a6b640 1 -- start start 2023-11-24T06:16:48.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.621+0000 7f1724a6b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1720070470 0x7f172006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.621+0000 7f1724a6b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f172006ec60 0x7f17200cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.622+0000 7f1724a6b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1720142d50 con 0x7f17200cd150 2023-11-24T06:16:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.622+0000 7f1724a6b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17200cd530 con 0x7f172006ec60 2023-11-24T06:16:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.622+0000 7f1724a6b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172006f430 con 0x7f1720070470 2023-11-24T06:16:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.622+0000 7f171ed9d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f172006ec60 0x7f17200cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.622+0000 7f171dd9b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1720070470 0x7f172006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.622+0000 7f171ed9d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f172006ec60 0x7f17200cb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:56138/0 (socket says 172.21.15.5:56138) 2023-11-24T06:16:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.622+0000 7f171ed9d640 1 -- 172.21.15.5:0/3322563 learned_addr learned my addr 172.21.15.5:0/3322563 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.623+0000 7f171ed9d640 1 -- 172.21.15.5:0/3322563 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1720070470 msgr2=0x7f172006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.623+0000 7f171ed9d640 1 --2- 172.21.15.5:0/3322563 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1720070470 0x7f172006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.623+0000 7f171ed9d640 1 -- 172.21.15.5:0/3322563 >> v1:172.21.15.6:6789/0 conn(0x7f17200cd150 legacy=0x7f172006fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.623+0000 7f171ed9d640 1 -- 172.21.15.5:0/3322563 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1720142fa0 con 0x7f172006ec60 2023-11-24T06:16:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.625+0000 7f171ed9d640 1 --2- 172.21.15.5:0/3322563 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f172006ec60 0x7f17200cb6f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1718009870 tx=0x7f171802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e67c71314de889b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.625+0000 7f171d59a640 1 -- 172.21.15.5:0/3322563 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718031640 con 0x7f172006ec60 2023-11-24T06:16:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.625+0000 7f171d59a640 1 -- 172.21.15.5:0/3322563 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17180317a0 con 0x7f172006ec60 2023-11-24T06:16:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.625+0000 7f171d59a640 1 -- 172.21.15.5:0/3322563 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1718031b30 con 0x7f172006ec60 2023-11-24T06:16:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.626+0000 7f1724a6b640 1 -- 172.21.15.5:0/3322563 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f172006ec60 msgr2=0x7f17200cb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.626+0000 7f1724a6b640 1 --2- 172.21.15.5:0/3322563 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f172006ec60 0x7f17200cb6f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1718009870 tx=0x7f171802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.626+0000 7f1724a6b640 1 -- 172.21.15.5:0/3322563 shutdown_connections 2023-11-24T06:16:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.626+0000 7f1724a6b640 1 --2- 172.21.15.5:0/3322563 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f172006ec60 0x7f17200cb6f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.626+0000 7f1724a6b640 1 --2- 172.21.15.5:0/3322563 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1720070470 0x7f172006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.626+0000 7f1724a6b640 1 -- 172.21.15.5:0/3322563 >> 172.21.15.5:0/3322563 conn(0x7f17200cdb20 msgr2=0x7f17200cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.626+0000 7f1724a6b640 1 -- 172.21.15.5:0/3322563 shutdown_connections 2023-11-24T06:16:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.626+0000 7f1724a6b640 1 -- 172.21.15.5:0/3322563 wait complete. 2023-11-24T06:16:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.628+0000 7f1724a6b640 1 Processor -- start 2023-11-24T06:16:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.628+0000 7f1724a6b640 1 -- start start 2023-11-24T06:16:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.629+0000 7f1724a6b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f172006ec60 0x7f17200b6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.629+0000 7f1724a6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1720070470 0x7f1720156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.629+0000 7f1724a6b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17200cd150 0x7f17200c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.629+0000 7f1724a6b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17200c7770 con 0x7f1720070470 2023-11-24T06:16:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.629+0000 7f1724a6b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17200b6b70 con 0x7f17200cd150 2023-11-24T06:16:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.629+0000 7f1724a6b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1720156940 con 0x7f172006ec60 2023-11-24T06:16:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.629+0000 7f171dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1720070470 0x7f1720156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.629+0000 7f171dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1720070470 0x7f1720156400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40896/0 (socket says 172.21.15.5:40896) 2023-11-24T06:16:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.629+0000 7f171dd9b640 1 -- 172.21.15.5:0/3715307941 learned_addr learned my addr 172.21.15.5:0/3715307941 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.630+0000 7f171ed9d640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17200cd150 0x7f17200c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.630+0000 7f171e59c640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f172006ec60 0x7f17200b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.630+0000 7f171dd9b640 1 -- 172.21.15.5:0/3715307941 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f172006ec60 msgr2=0x7f17200b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.630+0000 7f171dd9b640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f172006ec60 0x7f17200b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.630+0000 7f171dd9b640 1 -- 172.21.15.5:0/3715307941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17200cd150 msgr2=0x7f17200c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.630+0000 7f171dd9b640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17200cd150 0x7f17200c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.630+0000 7f171dd9b640 1 -- 172.21.15.5:0/3715307941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17180094d0 con 0x7f1720070470 2023-11-24T06:16:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.630+0000 7f171ed9d640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17200cd150 0x7f17200c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.630+0000 7f171dd9b640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1720070470 0x7f1720156400 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f171400efc0 tx=0x7f171400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.630+0000 7f170effd640 1 -- 172.21.15.5:0/3715307941 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714009e80 con 0x7f1720070470 2023-11-24T06:16:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.630+0000 7f170effd640 1 -- 172.21.15.5:0/3715307941 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171400f040 con 0x7f1720070470 2023-11-24T06:16:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.631+0000 7f170effd640 1 -- 172.21.15.5:0/3715307941 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714014680 con 0x7f1720070470 2023-11-24T06:16:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.631+0000 7f1724a6b640 1 -- 172.21.15.5:0/3715307941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17200cb050 con 0x7f1720070470 2023-11-24T06:16:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.631+0000 7f1724a6b640 1 -- 172.21.15.5:0/3715307941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17200b4b00 con 0x7f1720070470 2023-11-24T06:16:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.632+0000 7f1724a6b640 1 -- 172.21.15.5:0/3715307941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172006fb50 con 0x7f1720070470 2023-11-24T06:16:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.633+0000 7f170effd640 1 -- 172.21.15.5:0/3715307941 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f17140147e0 con 0x7f1720070470 2023-11-24T06:16:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.633+0000 7f170effd640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f170006ae70 0x7f170006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.634+0000 7f170effd640 1 -- 172.21.15.5:0/3715307941 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f171408a580 con 0x7f1720070470 2023-11-24T06:16:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.634+0000 7f171e59c640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f170006ae70 0x7f170006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.634+0000 7f171e59c640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f170006ae70 0x7f170006d330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1710000f60 tx=0x7f1710009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.638+0000 7f170effd640 1 -- 172.21.15.5:0/3715307941 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1714055470 con 0x7f1720070470 2023-11-24T06:16:48.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.829+0000 7f1724a6b640 1 -- 172.21.15.5:0/3715307941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f172004e920 con 0x7f1720070470 2023-11-24T06:16:48.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.830+0000 7f170effd640 1 -- 172.21.15.5:0/3715307941 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1714058a90 con 0x7f1720070470 2023-11-24T06:16:48.833 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.832+0000 7f1724a6b640 1 -- 172.21.15.5:0/3715307941 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f170006ae70 msgr2=0x7f170006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.832+0000 7f1724a6b640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f170006ae70 0x7f170006d330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1710000f60 tx=0x7f1710009040 comp rx=0 tx=0).stop 2023-11-24T06:16:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.832+0000 7f1724a6b640 1 -- 172.21.15.5:0/3715307941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1720070470 msgr2=0x7f1720156400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.832+0000 7f1724a6b640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1720070470 0x7f1720156400 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f171400efc0 tx=0x7f171400c490 comp rx=0 tx=0).stop 2023-11-24T06:16:48.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.832+0000 7f1724a6b640 1 -- 172.21.15.5:0/3715307941 shutdown_connections 2023-11-24T06:16:48.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.832+0000 7f1724a6b640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f170006ae70 0x7f170006d330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.832+0000 7f1724a6b640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f17200cd150 0x7f17200c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.832+0000 7f1724a6b640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1720070470 0x7f1720156400 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.832+0000 7f1724a6b640 1 --2- 172.21.15.5:0/3715307941 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f172006ec60 0x7f17200b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:48.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.832+0000 7f1724a6b640 1 -- 172.21.15.5:0/3715307941 >> 172.21.15.5:0/3715307941 conn(0x7f17200cdb20 msgr2=0x7f172006c730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:48.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.833+0000 7f1724a6b640 1 -- 172.21.15.5:0/3715307941 shutdown_connections 2023-11-24T06:16:48.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:48.833+0000 7f1724a6b640 1 -- 172.21.15.5:0/3715307941 wait complete. 2023-11-24T06:16:48.855 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:16:49.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.080+0000 7f8424a71640 1 Processor -- start 2023-11-24T06:16:49.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.080+0000 7f8424a71640 1 -- start start 2023-11-24T06:16:49.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.082+0000 7f8424a71640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842015f460 0x7f842006cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:49.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.082+0000 7f8424a71640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8420141e80 0x7f842006d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:49.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.082+0000 7f8424a71640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8420070bf0 con 0x7f84201416f0 2023-11-24T06:16:49.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.082+0000 7f8424a71640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f842006db30 con 0x7f8420141e80 2023-11-24T06:16:49.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.082+0000 7f8424a71640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842006d5a0 con 0x7f842015f460 2023-11-24T06:16:49.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.082+0000 7f841ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8420141e80 0x7f842006d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:49.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.082+0000 7f841ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8420141e80 0x7f842006d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40910/0 (socket says 172.21.15.5:40910) 2023-11-24T06:16:49.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.082+0000 7f841ffff640 1 -- 172.21.15.5:0/17162139 learned_addr learned my addr 172.21.15.5:0/17162139 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:49.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.082+0000 7f841ffff640 1 -- 172.21.15.5:0/17162139 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842015f460 msgr2=0x7f842006cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:49.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.082+0000 7f8417fff640 1 --2- 172.21.15.5:0/17162139 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842015f460 0x7f842006cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:49.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.082+0000 7f841ffff640 1 --2- 172.21.15.5:0/17162139 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842015f460 0x7f842006cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.083+0000 7f841ffff640 1 -- 172.21.15.5:0/17162139 >> v1:172.21.15.6:6789/0 conn(0x7f84201416f0 legacy=0x7f842015ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:49.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.083+0000 7f841ffff640 1 -- 172.21.15.5:0/17162139 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f842006d850 con 0x7f8420141e80 2023-11-24T06:16:49.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.083+0000 7f8417fff640 1 --2- 172.21.15.5:0/17162139 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842015f460 0x7f842006cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:49.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.084+0000 7f841ffff640 1 --2- 172.21.15.5:0/17162139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8420141e80 0x7f842006d060 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8418009e50 tx=0x7f841802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960a9086c7054179 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:49.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.084+0000 7f841effd640 1 -- 172.21.15.5:0/17162139 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841802ca80 con 0x7f8420141e80 2023-11-24T06:16:49.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.084+0000 7f841effd640 1 -- 172.21.15.5:0/17162139 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841802cbe0 con 0x7f8420141e80 2023-11-24T06:16:49.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.084+0000 7f841effd640 1 -- 172.21.15.5:0/17162139 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84180343f0 con 0x7f8420141e80 2023-11-24T06:16:49.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.085+0000 7f8424a71640 1 -- 172.21.15.5:0/17162139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8420141e80 msgr2=0x7f842006d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:49.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.085+0000 7f8424a71640 1 --2- 172.21.15.5:0/17162139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8420141e80 0x7f842006d060 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8418009e50 tx=0x7f841802bee0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.085+0000 7f8424a71640 1 -- 172.21.15.5:0/17162139 shutdown_connections 2023-11-24T06:16:49.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.085+0000 7f8424a71640 1 --2- 172.21.15.5:0/17162139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8420141e80 0x7f842006d060 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.085+0000 7f8424a71640 1 --2- 172.21.15.5:0/17162139 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f842015f460 0x7f842006cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.085+0000 7f8424a71640 1 -- 172.21.15.5:0/17162139 >> 172.21.15.5:0/17162139 conn(0x7f84200b0840 msgr2=0x7f84200b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:49.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.085+0000 7f8424a71640 1 -- 172.21.15.5:0/17162139 shutdown_connections 2023-11-24T06:16:49.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.085+0000 7f8424a71640 1 -- 172.21.15.5:0/17162139 wait complete. 2023-11-24T06:16:49.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.087+0000 7f8424a71640 1 Processor -- start 2023-11-24T06:16:49.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.087+0000 7f8424a71640 1 -- start start 2023-11-24T06:16:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.088+0000 7f8424a71640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84201416f0 0x7f84201f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.088+0000 7f8424a71640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8420141e80 0x7f842015b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.088+0000 7f8424a71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f842015f460 0x7f842015d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.088+0000 7f8424a71640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84201f73c0 con 0x7f842015f460 2023-11-24T06:16:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.088+0000 7f8424a71640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84200b1e70 con 0x7f84201416f0 2023-11-24T06:16:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.088+0000 7f8424a71640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842015bd90 con 0x7f8420141e80 2023-11-24T06:16:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.088+0000 7f841ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f842015f460 0x7f842015d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.088+0000 7f841ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f842015f460 0x7f842015d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40922/0 (socket says 172.21.15.5:40922) 2023-11-24T06:16:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.088+0000 7f841ffff640 1 -- 172.21.15.5:0/1697852843 learned_addr learned my addr 172.21.15.5:0/1697852843 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.088+0000 7f841f7fe640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84201416f0 0x7f84201f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.088+0000 7f8417fff640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8420141e80 0x7f842015b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.088+0000 7f841ffff640 1 -- 172.21.15.5:0/1697852843 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8420141e80 msgr2=0x7f842015b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.089+0000 7f841ffff640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8420141e80 0x7f842015b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.089+0000 7f841ffff640 1 -- 172.21.15.5:0/1697852843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84201416f0 msgr2=0x7f84201f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.089+0000 7f841ffff640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84201416f0 0x7f84201f6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.089+0000 7f841ffff640 1 -- 172.21.15.5:0/1697852843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8418009b30 con 0x7f842015f460 2023-11-24T06:16:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.089+0000 7f841f7fe640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84201416f0 0x7f84201f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:49.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.089+0000 7f841ffff640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f842015f460 0x7f842015d2e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8418036040 tx=0x7f8418004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:49.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.089+0000 7f841cff9640 1 -- 172.21.15.5:0/1697852843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841803e450 con 0x7f842015f460 2023-11-24T06:16:49.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.089+0000 7f841cff9640 1 -- 172.21.15.5:0/1697852843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841803e5d0 con 0x7f842015f460 2023-11-24T06:16:49.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.089+0000 7f841cff9640 1 -- 172.21.15.5:0/1697852843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841803e980 con 0x7f842015f460 2023-11-24T06:16:49.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.089+0000 7f8424a71640 1 -- 172.21.15.5:0/1697852843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84201f6b20 con 0x7f842015f460 2023-11-24T06:16:49.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.089+0000 7f8424a71640 1 -- 172.21.15.5:0/1697852843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84201f70e0 con 0x7f842015f460 2023-11-24T06:16:49.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.090+0000 7f8424a71640 1 -- 172.21.15.5:0/1697852843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84200b1c40 con 0x7f842015f460 2023-11-24T06:16:49.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.091+0000 7f841cff9640 1 -- 172.21.15.5:0/1697852843 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f841802cd50 con 0x7f842015f460 2023-11-24T06:16:49.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.092+0000 7f841cff9640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f840006ae70 0x7f840006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:49.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.092+0000 7f841cff9640 1 -- 172.21.15.5:0/1697852843 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f84180abfe0 con 0x7f842015f460 2023-11-24T06:16:49.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.093+0000 7f841f7fe640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f840006ae70 0x7f840006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.093+0000 7f841f7fe640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f840006ae70 0x7f840006d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8410003eb0 tx=0x7f8410009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.097+0000 7f841cff9640 1 -- 172.21.15.5:0/1697852843 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84200b1c40 con 0x7f842015f460 2023-11-24T06:16:49.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.297+0000 7f8424a71640 1 -- 172.21.15.5:0/1697852843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f84200b1c40 con 0x7f842015f460 2023-11-24T06:16:49.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.298+0000 7f841cff9640 1 -- 172.21.15.5:0/1697852843 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f84200b1c40 con 0x7f842015f460 2023-11-24T06:16:49.301 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2023-11-24T06:16:49.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.301+0000 7f8424a71640 1 -- 172.21.15.5:0/1697852843 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f840006ae70 msgr2=0x7f840006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:49.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.301+0000 7f8424a71640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f840006ae70 0x7f840006d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8410003eb0 tx=0x7f8410009040 comp rx=0 tx=0).stop 2023-11-24T06:16:49.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.301+0000 7f8424a71640 1 -- 172.21.15.5:0/1697852843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f842015f460 msgr2=0x7f842015d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:49.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.301+0000 7f8424a71640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f842015f460 0x7f842015d2e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8418036040 tx=0x7f8418004a40 comp rx=0 tx=0).stop 2023-11-24T06:16:49.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.301+0000 7f8424a71640 1 -- 172.21.15.5:0/1697852843 shutdown_connections 2023-11-24T06:16:49.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.301+0000 7f8424a71640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f840006ae70 0x7f840006d330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.301+0000 7f8424a71640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8420141e80 0x7f842015b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.301+0000 7f8424a71640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f842015f460 0x7f842015d2e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.301+0000 7f8424a71640 1 --2- 172.21.15.5:0/1697852843 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f84201416f0 0x7f84201f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.301+0000 7f8424a71640 1 -- 172.21.15.5:0/1697852843 >> 172.21.15.5:0/1697852843 conn(0x7f84200b0840 msgr2=0x7f84200ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:49.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.301+0000 7f8424a71640 1 -- 172.21.15.5:0/1697852843 shutdown_connections 2023-11-24T06:16:49.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.302+0000 7f8424a71640 1 -- 172.21.15.5:0/1697852843 wait complete. 2023-11-24T06:16:49.320 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi006.front.sepia.ceph.com... 2023-11-24T06:16:49.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:49.320 DEBUG:teuthology.orchestra.run.smithi006:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T06:16:49.434 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:16:49.435 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T06:16:49.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:49.435 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:16:49.524 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:16:49.525 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:16:49.585 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:16:49.585 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:16:49.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:49.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.900+0000 7ff62f347640 1 Processor -- start 2023-11-24T06:16:49.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.901+0000 7ff62f347640 1 -- start start 2023-11-24T06:16:49.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.902+0000 7ff62f347640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6280cf050 0x7ff6280cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:49.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.903+0000 7ff62f347640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff628140900 con 0x7ff62813dfd0 2023-11-24T06:16:49.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.903+0000 7ff62f347640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62815fa70 con 0x7ff6280cd7b0 2023-11-24T06:16:49.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.903+0000 7ff62f347640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62815c580 con 0x7ff6280cf050 2023-11-24T06:16:49.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.903+0000 7ff62e345640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff62813dfd0 0x7ff62815be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56216/0 (socket says 172.21.15.5:56216) 2023-11-24T06:16:49.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.903+0000 7ff62e345640 1 -- 172.21.15.5:0/3473984017 learned_addr learned my addr 172.21.15.5:0/3473984017 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:49.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.903+0000 7ff62d343640 1 -- 172.21.15.5:0/3473984017 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187190403 0 0) 0x7ff628140900 con 0x7ff62813dfd0 2023-11-24T06:16:49.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.903+0000 7ff62d343640 1 -- 172.21.15.5:0/3473984017 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff614003520 con 0x7ff62813dfd0 2023-11-24T06:16:49.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62db44640 1 --2- 172.21.15.5:0/3473984017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6280cf050 0x7ff6280cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:49.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62d343640 1 -- 172.21.15.5:0/3473984017 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3298121325 0 0) 0x7ff614003520 con 0x7ff62813dfd0 2023-11-24T06:16:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62d343640 1 -- 172.21.15.5:0/3473984017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6280cf050 msgr2=0x7ff6280cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62d343640 1 --2- 172.21.15.5:0/3473984017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6280cf050 0x7ff6280cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62d343640 1 -- 172.21.15.5:0/3473984017 >> v1:172.21.15.6:6789/0 conn(0x7ff6280cd7b0 legacy=0x7ff6280cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62d343640 1 -- 172.21.15.5:0/3473984017 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62815ee60 con 0x7ff62813dfd0 2023-11-24T06:16:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62d343640 1 -- 172.21.15.5:0/3473984017 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7ff624001670 con 0x7ff62813dfd0 2023-11-24T06:16:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62d343640 1 -- 172.21.15.5:0/3473984017 >> v1:172.21.15.5:6789/0 conn(0x7ff62813dfd0 legacy=0x7ff62815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62d343640 1 --2- 172.21.15.5:0/3473984017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff614003a10 0x7ff614003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62d343640 1 --2- 172.21.15.5:0/3473984017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6280cf050 0x7ff6140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62d343640 1 --2- 172.21.15.5:0/3473984017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff614004d50 0x7ff6140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62d343640 1 -- 172.21.15.5:0/3473984017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62815c580 con 0x7ff614003a10 2023-11-24T06:16:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62d343640 1 -- 172.21.15.5:0/3473984017 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61400bb60 con 0x7ff614004d50 2023-11-24T06:16:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62d343640 1 -- 172.21.15.5:0/3473984017 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61400bd20 con 0x7ff6280cf050 2023-11-24T06:16:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.904+0000 7ff62e345640 1 --2- 172.21.15.5:0/3473984017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff614003a10 0x7ff614003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.905+0000 7ff62db44640 1 --2- 172.21.15.5:0/3473984017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6280cf050 0x7ff6140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.905+0000 7ff62e345640 1 -- 172.21.15.5:0/3473984017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6280cf050 msgr2=0x7ff6140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.905+0000 7ff62eb46640 1 --2- 172.21.15.5:0/3473984017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff614004d50 0x7ff6140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.905+0000 7ff62e345640 1 --2- 172.21.15.5:0/3473984017 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6280cf050 0x7ff6140047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.905+0000 7ff62e345640 1 -- 172.21.15.5:0/3473984017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff614004d50 msgr2=0x7ff6140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.905+0000 7ff62e345640 1 --2- 172.21.15.5:0/3473984017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff614004d50 0x7ff6140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.905+0000 7ff62e345640 1 -- 172.21.15.5:0/3473984017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff614003520 con 0x7ff614003a10 2023-11-24T06:16:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.905+0000 7ff62eb46640 1 --2- 172.21.15.5:0/3473984017 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff614004d50 0x7ff6140051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:49.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.906+0000 7ff62e345640 1 --2- 172.21.15.5:0/3473984017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff614003a10 0x7ff614003e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff624007fd0 tx=0x7ff6140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f828ff46367eae91 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:49.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.906+0000 7ff62d343640 1 -- 172.21.15.5:0/3473984017 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624005ad0 con 0x7ff614003a10 2023-11-24T06:16:49.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.906+0000 7ff62d343640 1 -- 172.21.15.5:0/3473984017 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff624005c30 con 0x7ff614003a10 2023-11-24T06:16:49.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.906+0000 7ff62d343640 1 -- 172.21.15.5:0/3473984017 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62402c7c0 con 0x7ff614003a10 2023-11-24T06:16:49.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.907+0000 7ff62f347640 1 -- 172.21.15.5:0/3473984017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff614003a10 msgr2=0x7ff614003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:49.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.907+0000 7ff62f347640 1 --2- 172.21.15.5:0/3473984017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff614003a10 0x7ff614003e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff624007fd0 tx=0x7ff6140057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.907+0000 7ff62eb46640 1 -- 172.21.15.5:0/3473984017 reap_dead start 2023-11-24T06:16:49.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.907+0000 7ff62f347640 1 -- 172.21.15.5:0/3473984017 shutdown_connections 2023-11-24T06:16:49.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.907+0000 7ff62f347640 1 -- 172.21.15.5:0/3473984017 >> 172.21.15.5:0/3473984017 conn(0x7ff62806a860 msgr2=0x7ff62806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:49.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.907+0000 7ff62f347640 1 -- 172.21.15.5:0/3473984017 shutdown_connections 2023-11-24T06:16:49.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.907+0000 7ff62f347640 1 -- 172.21.15.5:0/3473984017 wait complete. 2023-11-24T06:16:49.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.909+0000 7ff62f347640 1 Processor -- start 2023-11-24T06:16:49.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.909+0000 7ff62f347640 1 -- start start 2023-11-24T06:16:49.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.910+0000 7ff62f347640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff614003a10 0x7ff6281585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:49.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.910+0000 7ff62f347640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6280c7ec0 0x7ff6281f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:49.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.910+0000 7ff62f347640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6281543d0 0x7ff628154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:49.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.910+0000 7ff62f347640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280c8430 con 0x7ff6280c7ec0 2023-11-24T06:16:49.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.910+0000 7ff62f347640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff628154dc0 con 0x7ff614003a10 2023-11-24T06:16:49.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.910+0000 7ff62f347640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62813d820 con 0x7ff6281543d0 2023-11-24T06:16:49.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.910+0000 7ff62db44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6280c7ec0 0x7ff6281f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:49.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.910+0000 7ff62db44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6280c7ec0 0x7ff6281f6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40942/0 (socket says 172.21.15.5:40942) 2023-11-24T06:16:49.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.910+0000 7ff62db44640 1 -- 172.21.15.5:0/2430094548 learned_addr learned my addr 172.21.15.5:0/2430094548 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:49.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.911+0000 7ff62e345640 1 --2- 172.21.15.5:0/2430094548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff614003a10 0x7ff6281585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:49.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.911+0000 7ff62db44640 1 -- 172.21.15.5:0/2430094548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6281543d0 msgr2=0x7ff628154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:49.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.911+0000 7ff62db44640 1 --2- 172.21.15.5:0/2430094548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6281543d0 0x7ff628154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.911+0000 7ff62db44640 1 -- 172.21.15.5:0/2430094548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff614003a10 msgr2=0x7ff6281585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:49.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.911+0000 7ff62db44640 1 --2- 172.21.15.5:0/2430094548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff614003a10 0x7ff6281585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:49.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.911+0000 7ff62db44640 1 -- 172.21.15.5:0/2430094548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff628159000 con 0x7ff6280c7ec0 2023-11-24T06:16:49.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.911+0000 7ff62db44640 1 --2- 172.21.15.5:0/2430094548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6280c7ec0 0x7ff6281f6870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff61800f580 tx=0x7ff61800fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:49.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.911+0000 7ff61effd640 1 -- 172.21.15.5:0/2430094548 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618014870 con 0x7ff6280c7ec0 2023-11-24T06:16:49.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.911+0000 7ff61effd640 1 -- 172.21.15.5:0/2430094548 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6180149d0 con 0x7ff6280c7ec0 2023-11-24T06:16:49.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.911+0000 7ff62f347640 1 -- 172.21.15.5:0/2430094548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6281592e0 con 0x7ff6280c7ec0 2023-11-24T06:16:49.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.911+0000 7ff61effd640 1 -- 172.21.15.5:0/2430094548 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618014d80 con 0x7ff6280c7ec0 2023-11-24T06:16:49.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.912+0000 7ff62f347640 1 -- 172.21.15.5:0/2430094548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6281598a0 con 0x7ff6280c7ec0 2023-11-24T06:16:49.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.913+0000 7ff62f347640 1 -- 172.21.15.5:0/2430094548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62806d530 con 0x7ff6280c7ec0 2023-11-24T06:16:49.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.914+0000 7ff61effd640 1 -- 172.21.15.5:0/2430094548 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff618015410 con 0x7ff6280c7ec0 2023-11-24T06:16:49.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.915+0000 7ff61effd640 1 --2- 172.21.15.5:0/2430094548 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff60c06ae40 0x7ff60c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:49.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.915+0000 7ff61effd640 1 -- 172.21.15.5:0/2430094548 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff61808a050 con 0x7ff6280c7ec0 2023-11-24T06:16:49.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.915+0000 7ff62e345640 1 --2- 172.21.15.5:0/2430094548 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff60c06ae40 0x7ff60c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:49.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.916+0000 7ff62e345640 1 --2- 172.21.15.5:0/2430094548 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff60c06ae40 0x7ff60c06d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff624034000 tx=0x7ff624009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:49.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:49.919+0000 7ff61effd640 1 -- 172.21.15.5:0/2430094548 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff618055080 con 0x7ff6280c7ec0 2023-11-24T06:16:50.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.115+0000 7ff62f347640 1 -- 172.21.15.5:0/2430094548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff62804e840 con 0x7ff6280c7ec0 2023-11-24T06:16:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.118+0000 7ff61effd640 1 -- 172.21.15.5:0/2430094548 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16658 (secure 0 0 0) 0x7ff6180586a0 con 0x7ff6280c7ec0 2023-11-24T06:16:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:50.122 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:16:33.958286+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:50.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.121+0000 7ff62f347640 1 -- 172.21.15.5:0/2430094548 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff60c06ae40 msgr2=0x7ff60c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:50.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.121+0000 7ff62f347640 1 --2- 172.21.15.5:0/2430094548 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff60c06ae40 0x7ff60c06d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff624034000 tx=0x7ff624009030 comp rx=0 tx=0).stop 2023-11-24T06:16:50.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.121+0000 7ff62f347640 1 -- 172.21.15.5:0/2430094548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6280c7ec0 msgr2=0x7ff6281f6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:50.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.121+0000 7ff62f347640 1 --2- 172.21.15.5:0/2430094548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6280c7ec0 0x7ff6281f6870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff61800f580 tx=0x7ff61800fa40 comp rx=0 tx=0).stop 2023-11-24T06:16:50.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.121+0000 7ff62f347640 1 -- 172.21.15.5:0/2430094548 shutdown_connections 2023-11-24T06:16:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.121+0000 7ff62f347640 1 --2- 172.21.15.5:0/2430094548 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff60c06ae40 0x7ff60c06d300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.121+0000 7ff62f347640 1 --2- 172.21.15.5:0/2430094548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6281543d0 0x7ff628154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.121+0000 7ff62f347640 1 --2- 172.21.15.5:0/2430094548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6280c7ec0 0x7ff6281f6870 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.121+0000 7ff62f347640 1 --2- 172.21.15.5:0/2430094548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff614003a10 0x7ff6281585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.121+0000 7ff62f347640 1 -- 172.21.15.5:0/2430094548 >> 172.21.15.5:0/2430094548 conn(0x7ff62806a860 msgr2=0x7ff628071900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.121+0000 7ff62f347640 1 -- 172.21.15.5:0/2430094548 shutdown_connections 2023-11-24T06:16:50.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.121+0000 7ff62f347640 1 -- 172.21.15.5:0/2430094548 wait complete. 2023-11-24T06:16:50.146 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:50.146 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:50.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.374+0000 7fe0435ce640 1 Processor -- start 2023-11-24T06:16:50.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.374+0000 7fe0435ce640 1 -- start start 2023-11-24T06:16:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.376+0000 7fe0435ce640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe03c160b10 0x7fe03c06e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.376+0000 7fe0435ce640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03c071780 con 0x7fe03c072280 2023-11-24T06:16:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.376+0000 7fe0435ce640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03c0711c0 con 0x7fe03c06cad0 2023-11-24T06:16:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.376+0000 7fe0435ce640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03c0b2320 con 0x7fe03c160b10 2023-11-24T06:16:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.376+0000 7fe040b69640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe03c072280 0x7fe03c06c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56224/0 (socket says 172.21.15.5:56224) 2023-11-24T06:16:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.376+0000 7fe04136a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe03c160b10 0x7fe03c06e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:50.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.376+0000 7fe040b69640 1 -- 172.21.15.5:0/1699959113 learned_addr learned my addr 172.21.15.5:0/1699959113 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.376+0000 7fe033fff640 1 -- 172.21.15.5:0/1699959113 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081779626 0 0) 0x7fe03c071780 con 0x7fe03c072280 2023-11-24T06:16:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.376+0000 7fe033fff640 1 -- 172.21.15.5:0/1699959113 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe028003520 con 0x7fe03c072280 2023-11-24T06:16:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.377+0000 7fe04136a640 1 -- 172.21.15.5:0/1699959113 >> v1:172.21.15.6:6789/0 conn(0x7fe03c06cad0 legacy=0x7fe03c0b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.377+0000 7fe04136a640 1 -- 172.21.15.5:0/1699959113 >> v1:172.21.15.5:6789/0 conn(0x7fe03c072280 legacy=0x7fe03c06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:50.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.377+0000 7fe04136a640 1 -- 172.21.15.5:0/1699959113 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c0714a0 con 0x7fe03c160b10 2023-11-24T06:16:50.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.378+0000 7fe04136a640 1 --2- 172.21.15.5:0/1699959113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe03c160b10 0x7fe03c06e420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe0380097a0 tx=0x7fe03802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88651fcd2cb98b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:50.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.378+0000 7fe033fff640 1 -- 172.21.15.5:0/1699959113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03802e070 con 0x7fe03c160b10 2023-11-24T06:16:50.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.378+0000 7fe033fff640 1 -- 172.21.15.5:0/1699959113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0380315a0 con 0x7fe03c160b10 2023-11-24T06:16:50.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.378+0000 7fe033fff640 1 -- 172.21.15.5:0/1699959113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038031930 con 0x7fe03c160b10 2023-11-24T06:16:50.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.379+0000 7fe0435ce640 1 -- 172.21.15.5:0/1699959113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe03c160b10 msgr2=0x7fe03c06e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:50.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.379+0000 7fe0435ce640 1 --2- 172.21.15.5:0/1699959113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe03c160b10 0x7fe03c06e420 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe0380097a0 tx=0x7fe03802b840 comp rx=0 tx=0).stop 2023-11-24T06:16:50.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.379+0000 7fe0435ce640 1 -- 172.21.15.5:0/1699959113 shutdown_connections 2023-11-24T06:16:50.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.379+0000 7fe0435ce640 1 --2- 172.21.15.5:0/1699959113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe03c160b10 0x7fe03c06e420 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.379+0000 7fe0435ce640 1 -- 172.21.15.5:0/1699959113 >> 172.21.15.5:0/1699959113 conn(0x7fe03c0b0ba0 msgr2=0x7fe03c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:50.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.379+0000 7fe0435ce640 1 -- 172.21.15.5:0/1699959113 shutdown_connections 2023-11-24T06:16:50.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.379+0000 7fe0435ce640 1 -- 172.21.15.5:0/1699959113 wait complete. 2023-11-24T06:16:50.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.381+0000 7fe0435ce640 1 Processor -- start 2023-11-24T06:16:50.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.381+0000 7fe0435ce640 1 -- start start 2023-11-24T06:16:50.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.382+0000 7fe0435ce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe03c06cad0 0x7fe03c0c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:50.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.382+0000 7fe0435ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe03c072280 0x7fe03c14c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:50.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.382+0000 7fe0435ce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe03c160b10 0x7fe03c0afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:50.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.382+0000 7fe0435ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c153230 con 0x7fe03c072280 2023-11-24T06:16:50.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.382+0000 7fe0435ce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c0cb360 con 0x7fe03c06cad0 2023-11-24T06:16:50.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.382+0000 7fe0435ce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c0cada0 con 0x7fe03c160b10 2023-11-24T06:16:50.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.382+0000 7fe040b69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe03c072280 0x7fe03c14c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:50.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.382+0000 7fe040b69640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe03c072280 0x7fe03c14c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40962/0 (socket says 172.21.15.5:40962) 2023-11-24T06:16:50.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.382+0000 7fe040b69640 1 -- 172.21.15.5:0/2163612643 learned_addr learned my addr 172.21.15.5:0/2163612643 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.382+0000 7fe04136a640 1 --2- 172.21.15.5:0/2163612643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe03c06cad0 0x7fe03c0c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.382+0000 7fe041b6b640 1 --2- 172.21.15.5:0/2163612643 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe03c160b10 0x7fe03c0afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.383+0000 7fe040b69640 1 -- 172.21.15.5:0/2163612643 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe03c160b10 msgr2=0x7fe03c0afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.383+0000 7fe040b69640 1 --2- 172.21.15.5:0/2163612643 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe03c160b10 0x7fe03c0afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.383+0000 7fe040b69640 1 -- 172.21.15.5:0/2163612643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe03c06cad0 msgr2=0x7fe03c0c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:50.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.383+0000 7fe040b69640 1 --2- 172.21.15.5:0/2163612643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe03c06cad0 0x7fe03c0c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.383+0000 7fe040b69640 1 -- 172.21.15.5:0/2163612643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe038009400 con 0x7fe03c072280 2023-11-24T06:16:50.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.383+0000 7fe040b69640 1 --2- 172.21.15.5:0/2163612643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe03c072280 0x7fe03c14c460 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00c500 tx=0x7fe02c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:50.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.383+0000 7fe031ffb640 1 -- 172.21.15.5:0/2163612643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c005ce0 con 0x7fe03c072280 2023-11-24T06:16:50.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.383+0000 7fe031ffb640 1 -- 172.21.15.5:0/2163612643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02c005e40 con 0x7fe03c072280 2023-11-24T06:16:50.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.383+0000 7fe031ffb640 1 -- 172.21.15.5:0/2163612643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c007330 con 0x7fe03c072280 2023-11-24T06:16:50.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.383+0000 7fe0435ce640 1 -- 172.21.15.5:0/2163612643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03c0cb080 con 0x7fe03c072280 2023-11-24T06:16:50.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.383+0000 7fe0435ce640 1 -- 172.21.15.5:0/2163612643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03c150640 con 0x7fe03c072280 2023-11-24T06:16:50.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.385+0000 7fe0435ce640 1 -- 172.21.15.5:0/2163612643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03c150830 con 0x7fe03c072280 2023-11-24T06:16:50.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.386+0000 7fe031ffb640 1 -- 172.21.15.5:0/2163612643 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe02c007550 con 0x7fe03c072280 2023-11-24T06:16:50.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.386+0000 7fe031ffb640 1 --2- 172.21.15.5:0/2163612643 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe02006ae40 0x7fe02006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:50.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.387+0000 7fe031ffb640 1 -- 172.21.15.5:0/2163612643 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe02c088d50 con 0x7fe03c072280 2023-11-24T06:16:50.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.387+0000 7fe04136a640 1 --2- 172.21.15.5:0/2163612643 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe02006ae40 0x7fe02006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:50.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.388+0000 7fe04136a640 1 --2- 172.21.15.5:0/2163612643 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe02006ae40 0x7fe02006d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe038036000 tx=0x7fe03803d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:50.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.391+0000 7fe031ffb640 1 -- 172.21.15.5:0/2163612643 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe02c053e20 con 0x7fe03c072280 2023-11-24T06:16:50.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.585+0000 7fe0435ce640 1 -- 172.21.15.5:0/2163612643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe03c06df00 con 0x7fe03c072280 2023-11-24T06:16:50.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.586+0000 7fe031ffb640 1 -- 172.21.15.5:0/2163612643 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe02c017020 con 0x7fe03c072280 2023-11-24T06:16:50.589 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.589+0000 7fe0435ce640 1 -- 172.21.15.5:0/2163612643 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe02006ae40 msgr2=0x7fe02006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.589+0000 7fe0435ce640 1 --2- 172.21.15.5:0/2163612643 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe02006ae40 0x7fe02006d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe038036000 tx=0x7fe03803d040 comp rx=0 tx=0).stop 2023-11-24T06:16:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.589+0000 7fe0435ce640 1 -- 172.21.15.5:0/2163612643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe03c072280 msgr2=0x7fe03c14c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:50.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.589+0000 7fe0435ce640 1 --2- 172.21.15.5:0/2163612643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe03c072280 0x7fe03c14c460 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00c500 tx=0x7fe02c00c9c0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.589+0000 7fe0435ce640 1 -- 172.21.15.5:0/2163612643 shutdown_connections 2023-11-24T06:16:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.589+0000 7fe0435ce640 1 --2- 172.21.15.5:0/2163612643 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe02006ae40 0x7fe02006d300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.589+0000 7fe0435ce640 1 --2- 172.21.15.5:0/2163612643 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe03c160b10 0x7fe03c0afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.589+0000 7fe0435ce640 1 --2- 172.21.15.5:0/2163612643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe03c072280 0x7fe03c14c460 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.589+0000 7fe0435ce640 1 --2- 172.21.15.5:0/2163612643 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe03c06cad0 0x7fe03c0c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.589+0000 7fe0435ce640 1 -- 172.21.15.5:0/2163612643 >> 172.21.15.5:0/2163612643 conn(0x7fe03c0b0ba0 msgr2=0x7fe03c15fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:50.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.590+0000 7fe0435ce640 1 -- 172.21.15.5:0/2163612643 shutdown_connections 2023-11-24T06:16:50.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.590+0000 7fe0435ce640 1 -- 172.21.15.5:0/2163612643 wait complete. 2023-11-24T06:16:50.613 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.843+0000 7f0a11f2e640 1 Processor -- start 2023-11-24T06:16:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.843+0000 7f0a11f2e640 1 -- start start 2023-11-24T06:16:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.844+0000 7f0a11f2e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0a0c0d1d90 0x7f0a0c0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.844+0000 7f0a11f2e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0a0c0d0fa0 0x7f0a0c0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.844+0000 7f0a11f2e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0c0c6bf0 con 0x7f0a0c15d3b0 2023-11-24T06:16:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.844+0000 7f0a11f2e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0c0d13f0 con 0x7f0a0c0d1d90 2023-11-24T06:16:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.844+0000 7f0a11f2e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0cedc0 con 0x7f0a0c0d0fa0 2023-11-24T06:16:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.845+0000 7f0a10f2c640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f0a0c15d3b0 0x7f0a0c0d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.5:58692/0 (socket says 172.21.15.5:58692) 2023-11-24T06:16:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.845+0000 7f0a10f2c640 1 -- 172.21.15.5:0/247231516 learned_addr learned my addr 172.21.15.5:0/247231516 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.845+0000 7f0a1172d640 1 --2- 172.21.15.5:0/247231516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0a0c0d0fa0 0x7f0a0c0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.845+0000 7f0a0bfff640 1 --2- 172.21.15.5:0/247231516 >> v2:172.21.15.6:3300/0 conn(0x7f0a0c0d1d90 0x7f0a0c0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.846+0000 7f0a1172d640 1 -- 172.21.15.5:0/247231516 >> v2:172.21.15.6:3300/0 conn(0x7f0a0c0d1d90 msgr2=0x7f0a0c0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.846+0000 7f0a1172d640 1 --2- 172.21.15.5:0/247231516 >> v2:172.21.15.6:3300/0 conn(0x7f0a0c0d1d90 0x7f0a0c0ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.846+0000 7f0a1172d640 1 -- 172.21.15.5:0/247231516 >> v1:172.21.15.6:6789/0 conn(0x7f0a0c15d3b0 legacy=0x7f0a0c0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.846+0000 7f0a0b7fe640 1 -- 172.21.15.5:0/247231516 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184977742 0 0) 0x7f0a0c0c6bf0 con 0x7f0a0c15d3b0 2023-11-24T06:16:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.846+0000 7f0a1172d640 1 -- 172.21.15.5:0/247231516 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0c6e40 con 0x7f0a0c0d0fa0 2023-11-24T06:16:50.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.847+0000 7f0a1172d640 1 --2- 172.21.15.5:0/247231516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0a0c0d0fa0 0x7f0a0c0b3ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0a040097a0 tx=0x7f0a0402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31f698f0a18bfd91 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.848+0000 7f0a0b7fe640 1 -- 172.21.15.5:0/247231516 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0402e070 con 0x7f0a0c0d0fa0 2023-11-24T06:16:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.848+0000 7f0a0b7fe640 1 -- 172.21.15.5:0/247231516 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a040315a0 con 0x7f0a0c0d0fa0 2023-11-24T06:16:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.848+0000 7f0a0b7fe640 1 -- 172.21.15.5:0/247231516 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04031930 con 0x7f0a0c0d0fa0 2023-11-24T06:16:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.848+0000 7f0a11f2e640 1 -- 172.21.15.5:0/247231516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0a0c0d0fa0 msgr2=0x7f0a0c0b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.848+0000 7f0a11f2e640 1 --2- 172.21.15.5:0/247231516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0a0c0d0fa0 0x7f0a0c0b3ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0a040097a0 tx=0x7f0a0402b840 comp rx=0 tx=0).stop 2023-11-24T06:16:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.849+0000 7f0a11f2e640 1 -- 172.21.15.5:0/247231516 shutdown_connections 2023-11-24T06:16:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.849+0000 7f0a11f2e640 1 --2- 172.21.15.5:0/247231516 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0a0c0d0fa0 0x7f0a0c0b3ca0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.849+0000 7f0a11f2e640 1 --2- 172.21.15.5:0/247231516 >> v2:172.21.15.6:3300/0 conn(0x7f0a0c0d1d90 0x7f0a0c0ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.849+0000 7f0a11f2e640 1 -- 172.21.15.5:0/247231516 >> 172.21.15.5:0/247231516 conn(0x7f0a0c0d30a0 msgr2=0x7f0a0c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.849+0000 7f0a11f2e640 1 -- 172.21.15.5:0/247231516 shutdown_connections 2023-11-24T06:16:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.849+0000 7f0a11f2e640 1 -- 172.21.15.5:0/247231516 wait complete. 2023-11-24T06:16:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.850+0000 7f0a11f2e640 1 Processor -- start 2023-11-24T06:16:50.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.851+0000 7f0a11f2e640 1 -- start start 2023-11-24T06:16:50.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.852+0000 7f0a11f2e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0a0c0d0fa0 0x7f0a0c1f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:50.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.852+0000 7f0a11f2e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a0c0d1d90 0x7f0a0c0ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:50.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.852+0000 7f0a11f2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a0c15d3b0 0x7f0a0c0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:50.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.852+0000 7f0a11f2e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1f2700 con 0x7f0a0c15d3b0 2023-11-24T06:16:50.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.852+0000 7f0a11f2e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0cac30 con 0x7f0a0c0d1d90 2023-11-24T06:16:50.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.852+0000 7f0a11f2e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0caf10 con 0x7f0a0c0d0fa0 2023-11-24T06:16:50.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.852+0000 7f0a1172d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a0c15d3b0 0x7f0a0c0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:50.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.852+0000 7f0a1172d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a0c15d3b0 0x7f0a0c0cbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40974/0 (socket says 172.21.15.5:40974) 2023-11-24T06:16:50.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.852+0000 7f0a1172d640 1 -- 172.21.15.5:0/1654237964 learned_addr learned my addr 172.21.15.5:0/1654237964 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:50.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.852+0000 7f0a0bfff640 1 --2- 172.21.15.5:0/1654237964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a0c0d1d90 0x7f0a0c0ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:50.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.852+0000 7f0a10f2c640 1 --2- 172.21.15.5:0/1654237964 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0a0c0d0fa0 0x7f0a0c1f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:50.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.853+0000 7f0a1172d640 1 -- 172.21.15.5:0/1654237964 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0a0c0d0fa0 msgr2=0x7f0a0c1f21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:50.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.853+0000 7f0a1172d640 1 --2- 172.21.15.5:0/1654237964 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0a0c0d0fa0 0x7f0a0c1f21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.853+0000 7f0a1172d640 1 -- 172.21.15.5:0/1654237964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a0c0d1d90 msgr2=0x7f0a0c0ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:50.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.853+0000 7f0a1172d640 1 --2- 172.21.15.5:0/1654237964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a0c0d1d90 0x7f0a0c0ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:50.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.853+0000 7f0a1172d640 1 -- 172.21.15.5:0/1654237964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a04009400 con 0x7f0a0c15d3b0 2023-11-24T06:16:50.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.853+0000 7f0a1172d640 1 --2- 172.21.15.5:0/1654237964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a0c15d3b0 0x7f0a0c0cbea0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0a04034040 tx=0x7f0a040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:50.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.853+0000 7f0a097fa640 1 -- 172.21.15.5:0/1654237964 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04032840 con 0x7f0a0c15d3b0 2023-11-24T06:16:50.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.853+0000 7f0a097fa640 1 -- 172.21.15.5:0/1654237964 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a040329a0 con 0x7f0a0c15d3b0 2023-11-24T06:16:50.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.853+0000 7f0a097fa640 1 -- 172.21.15.5:0/1654237964 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04032d50 con 0x7f0a0c15d3b0 2023-11-24T06:16:50.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.853+0000 7f0a11f2e640 1 -- 172.21.15.5:0/1654237964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c0cb1f0 con 0x7f0a0c15d3b0 2023-11-24T06:16:50.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.853+0000 7f0a11f2e640 1 -- 172.21.15.5:0/1654237964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c0cb7b0 con 0x7f0a0c15d3b0 2023-11-24T06:16:50.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.855+0000 7f0a11f2e640 1 -- 172.21.15.5:0/1654237964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0c0b5c60 con 0x7f0a0c15d3b0 2023-11-24T06:16:50.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.856+0000 7f0a097fa640 1 -- 172.21.15.5:0/1654237964 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0a0404d3f0 con 0x7f0a0c15d3b0 2023-11-24T06:16:50.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.857+0000 7f0a097fa640 1 --2- 172.21.15.5:0/1654237964 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f09ec06ae40 0x7f09ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:50.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.857+0000 7f0a097fa640 1 -- 172.21.15.5:0/1654237964 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0a040ab1d0 con 0x7f0a0c15d3b0 2023-11-24T06:16:50.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.857+0000 7f0a10f2c640 1 --2- 172.21.15.5:0/1654237964 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f09ec06ae40 0x7f09ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:50.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.858+0000 7f0a10f2c640 1 --2- 172.21.15.5:0/1654237964 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f09ec06ae40 0x7f09ec06d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f09fc007ed0 tx=0x7f09fc007e60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:50.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:50.862+0000 7f0a097fa640 1 -- 172.21.15.5:0/1654237964 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a0c0b5c60 con 0x7f0a0c15d3b0 2023-11-24T06:16:51.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.054+0000 7f0a11f2e640 1 -- 172.21.15.5:0/1654237964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a0c0b5c60 con 0x7f0a0c15d3b0 2023-11-24T06:16:51.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.055+0000 7f0a097fa640 1 -- 172.21.15.5:0/1654237964 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0a0c0b5c60 con 0x7f0a0c15d3b0 2023-11-24T06:16:51.058 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:51.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.057+0000 7f0a11f2e640 1 -- 172.21.15.5:0/1654237964 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f09ec06ae40 msgr2=0x7f09ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:51.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.057+0000 7f0a11f2e640 1 --2- 172.21.15.5:0/1654237964 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f09ec06ae40 0x7f09ec06d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f09fc007ed0 tx=0x7f09fc007e60 comp rx=0 tx=0).stop 2023-11-24T06:16:51.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.057+0000 7f0a11f2e640 1 -- 172.21.15.5:0/1654237964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a0c15d3b0 msgr2=0x7f0a0c0cbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:51.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.057+0000 7f0a11f2e640 1 --2- 172.21.15.5:0/1654237964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a0c15d3b0 0x7f0a0c0cbea0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0a04034040 tx=0x7f0a040046c0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.058+0000 7f0a11f2e640 1 -- 172.21.15.5:0/1654237964 shutdown_connections 2023-11-24T06:16:51.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.058+0000 7f0a11f2e640 1 --2- 172.21.15.5:0/1654237964 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f09ec06ae40 0x7f09ec06d300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.058+0000 7f0a11f2e640 1 --2- 172.21.15.5:0/1654237964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a0c15d3b0 0x7f0a0c0cbea0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.058+0000 7f0a11f2e640 1 --2- 172.21.15.5:0/1654237964 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0a0c0d1d90 0x7f0a0c0ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.058+0000 7f0a11f2e640 1 --2- 172.21.15.5:0/1654237964 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0a0c0d0fa0 0x7f0a0c1f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.058+0000 7f0a11f2e640 1 -- 172.21.15.5:0/1654237964 >> 172.21.15.5:0/1654237964 conn(0x7f0a0c0d30a0 msgr2=0x7f0a0c0d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:51.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.058+0000 7f0a11f2e640 1 -- 172.21.15.5:0/1654237964 shutdown_connections 2023-11-24T06:16:51.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.058+0000 7f0a11f2e640 1 -- 172.21.15.5:0/1654237964 wait complete. 2023-11-24T06:16:51.081 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:51.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.308+0000 7ff2d5fc1640 1 Processor -- start 2023-11-24T06:16:51.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.308+0000 7ff2d5fc1640 1 -- start start 2023-11-24T06:16:51.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.310+0000 7ff2d5fc1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff2d015a840 0x7ff2d00708f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:51.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.310+0000 7ff2d5fc1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2d0070e30 0x7ff2d006cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:51.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.310+0000 7ff2d5fc1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d00b23d0 con 0x7ff2d0156eb0 2023-11-24T06:16:51.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.310+0000 7ff2d5fc1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d00b2bf0 con 0x7ff2d015a840 2023-11-24T06:16:51.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.310+0000 7ff2d5fc1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d00b2630 con 0x7ff2d0070e30 2023-11-24T06:16:51.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.310+0000 7ff2d57c0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2d0070e30 0x7ff2d006cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:51.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.310+0000 7ff2cffff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff2d015a840 0x7ff2d00708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:51.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.311+0000 7ff2d57c0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2d0070e30 0x7ff2d006cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:39516/0 (socket says 172.21.15.5:39516) 2023-11-24T06:16:51.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.311+0000 7ff2d57c0640 1 -- 172.21.15.5:0/577210208 learned_addr learned my addr 172.21.15.5:0/577210208 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:51.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.311+0000 7ff2d57c0640 1 -- 172.21.15.5:0/577210208 >> v2:172.21.15.6:3300/0 conn(0x7ff2d015a840 msgr2=0x7ff2d00708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:51.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.311+0000 7ff2d57c0640 1 --2- 172.21.15.5:0/577210208 >> v2:172.21.15.6:3300/0 conn(0x7ff2d015a840 0x7ff2d00708f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.311+0000 7ff2d57c0640 1 -- 172.21.15.5:0/577210208 >> v1:172.21.15.6:6789/0 conn(0x7ff2d0156eb0 legacy=0x7ff2d00b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:51.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.311+0000 7ff2d57c0640 1 -- 172.21.15.5:0/577210208 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d00b2910 con 0x7ff2d0070e30 2023-11-24T06:16:51.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.311+0000 7ff2cffff640 1 --2- 172.21.15.5:0/577210208 >> v2:172.21.15.6:3300/0 conn(0x7ff2d015a840 0x7ff2d00708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:16:51.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.313+0000 7ff2d57c0640 1 --2- 172.21.15.5:0/577210208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2d0070e30 0x7ff2d006cba0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800d150 tx=0x7ff2c802e160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e08eabf3eb43a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:51.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.314+0000 7ff2cf7fe640 1 -- 172.21.15.5:0/577210208 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8038400 con 0x7ff2d0070e30 2023-11-24T06:16:51.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.314+0000 7ff2cf7fe640 1 -- 172.21.15.5:0/577210208 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c8032070 con 0x7ff2d0070e30 2023-11-24T06:16:51.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.314+0000 7ff2cf7fe640 1 -- 172.21.15.5:0/577210208 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8038780 con 0x7ff2d0070e30 2023-11-24T06:16:51.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.314+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/577210208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2d0070e30 msgr2=0x7ff2d006cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:51.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.314+0000 7ff2d5fc1640 1 --2- 172.21.15.5:0/577210208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2d0070e30 0x7ff2d006cba0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800d150 tx=0x7ff2c802e160 comp rx=0 tx=0).stop 2023-11-24T06:16:51.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.314+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/577210208 shutdown_connections 2023-11-24T06:16:51.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.314+0000 7ff2d5fc1640 1 --2- 172.21.15.5:0/577210208 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2d0070e30 0x7ff2d006cba0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.314+0000 7ff2d5fc1640 1 --2- 172.21.15.5:0/577210208 >> v2:172.21.15.6:3300/0 conn(0x7ff2d015a840 0x7ff2d00708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.314+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/577210208 >> 172.21.15.5:0/577210208 conn(0x7ff2d00cec10 msgr2=0x7ff2d00cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:51.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.315+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/577210208 shutdown_connections 2023-11-24T06:16:51.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.315+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/577210208 wait complete. 2023-11-24T06:16:51.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.316+0000 7ff2d5fc1640 1 Processor -- start 2023-11-24T06:16:51.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.317+0000 7ff2d5fc1640 1 -- start start 2023-11-24T06:16:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.318+0000 7ff2d5fc1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2d0156eb0 0x7ff2d01f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.318+0000 7ff2d5fc1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d015a840 0x7ff2d01f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.318+0000 7ff2d5fc1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2d01f39a0 0x7ff2d00af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.318+0000 7ff2d5fc1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d00b0340 con 0x7ff2d015a840 2023-11-24T06:16:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.318+0000 7ff2d5fc1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d01f6070 con 0x7ff2d01f39a0 2023-11-24T06:16:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.318+0000 7ff2d5fc1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d00b0060 con 0x7ff2d0156eb0 2023-11-24T06:16:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.318+0000 7ff2cffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d015a840 0x7ff2d01f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.318+0000 7ff2cffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d015a840 0x7ff2d01f5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40990/0 (socket says 172.21.15.5:40990) 2023-11-24T06:16:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.318+0000 7ff2cffff640 1 -- 172.21.15.5:0/788626780 learned_addr learned my addr 172.21.15.5:0/788626780 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.318+0000 7ff2d4fbf640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2d0156eb0 0x7ff2d01f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.319+0000 7ff2d57c0640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2d01f39a0 0x7ff2d00af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.319+0000 7ff2cffff640 1 -- 172.21.15.5:0/788626780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2d0156eb0 msgr2=0x7ff2d01f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.319+0000 7ff2cffff640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2d0156eb0 0x7ff2d01f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.319+0000 7ff2cffff640 1 -- 172.21.15.5:0/788626780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2d01f39a0 msgr2=0x7ff2d00af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.319+0000 7ff2cffff640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2d01f39a0 0x7ff2d00af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.319+0000 7ff2cffff640 1 -- 172.21.15.5:0/788626780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c0009400 con 0x7ff2d015a840 2023-11-24T06:16:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.319+0000 7ff2d57c0640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2d01f39a0 0x7ff2d00af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.319+0000 7ff2cffff640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d015a840 0x7ff2d01f5b30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000e960 tx=0x7ff2c000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.319+0000 7ff2cd7fa640 1 -- 172.21.15.5:0/788626780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c000ca00 con 0x7ff2d015a840 2023-11-24T06:16:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.319+0000 7ff2cd7fa640 1 -- 172.21.15.5:0/788626780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c000cb60 con 0x7ff2d015a840 2023-11-24T06:16:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.319+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/788626780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c800bd40 con 0x7ff2d015a840 2023-11-24T06:16:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.319+0000 7ff2cd7fa640 1 -- 172.21.15.5:0/788626780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0013470 con 0x7ff2d015a840 2023-11-24T06:16:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.320+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/788626780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d00cd440 con 0x7ff2d015a840 2023-11-24T06:16:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.321+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/788626780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d004e840 con 0x7ff2d015a840 2023-11-24T06:16:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.322+0000 7ff2cd7fa640 1 -- 172.21.15.5:0/788626780 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff2c00135d0 con 0x7ff2d015a840 2023-11-24T06:16:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.323+0000 7ff2cd7fa640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff2bc06ae40 0x7ff2bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.323+0000 7ff2cd7fa640 1 -- 172.21.15.5:0/788626780 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff2c0089d00 con 0x7ff2d015a840 2023-11-24T06:16:51.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.323+0000 7ff2d4fbf640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff2bc06ae40 0x7ff2bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:51.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.324+0000 7ff2d4fbf640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff2bc06ae40 0x7ff2bc06d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4000f60 tx=0x7ff2c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:51.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.327+0000 7ff2cd7fa640 1 -- 172.21.15.5:0/788626780 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff2c0054c90 con 0x7ff2d015a840 2023-11-24T06:16:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.520+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/788626780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff2d0157c10 con 0x7ff2d015a840 2023-11-24T06:16:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.521+0000 7ff2cd7fa640 1 -- 172.21.15.5:0/788626780 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff2c00582b0 con 0x7ff2d015a840 2023-11-24T06:16:51.524 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.524+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/788626780 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff2bc06ae40 msgr2=0x7ff2bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.524+0000 7ff2d5fc1640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff2bc06ae40 0x7ff2bc06d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4000f60 tx=0x7ff2c4009040 comp rx=0 tx=0).stop 2023-11-24T06:16:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.524+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/788626780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d015a840 msgr2=0x7ff2d01f5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.524+0000 7ff2d5fc1640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d015a840 0x7ff2d01f5b30 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff2c000e960 tx=0x7ff2c000ee20 comp rx=0 tx=0).stop 2023-11-24T06:16:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.524+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/788626780 shutdown_connections 2023-11-24T06:16:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.524+0000 7ff2d5fc1640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff2bc06ae40 0x7ff2bc06d300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.524+0000 7ff2d5fc1640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff2d01f39a0 0x7ff2d00af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.524+0000 7ff2d5fc1640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2d015a840 0x7ff2d01f5b30 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.524+0000 7ff2d5fc1640 1 --2- 172.21.15.5:0/788626780 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff2d0156eb0 0x7ff2d01f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.524+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/788626780 >> 172.21.15.5:0/788626780 conn(0x7ff2d00cec10 msgr2=0x7ff2d0155ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.524+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/788626780 shutdown_connections 2023-11-24T06:16:51.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.524+0000 7ff2d5fc1640 1 -- 172.21.15.5:0/788626780 wait complete. 2023-11-24T06:16:51.547 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:16:51.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.772+0000 7f0febffb640 1 Processor -- start 2023-11-24T06:16:51.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.772+0000 7f0febffb640 1 -- start start 2023-11-24T06:16:51.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.774+0000 7f0febffb640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0fe41460b0 0x7f0fe40b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:51.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.774+0000 7f0febffb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fe40b3780 0x7f0fe415d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.774+0000 7f0febffb640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe406cd50 con 0x7f0fe406b760 2023-11-24T06:16:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.774+0000 7f0febffb640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe4146a90 con 0x7f0fe41460b0 2023-11-24T06:16:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.774+0000 7f0febffb640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe4146490 con 0x7f0fe40b3780 2023-11-24T06:16:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.775+0000 7f0fe9596640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0fe406b760 0x7f0fe40b3070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56238/0 (socket says 172.21.15.5:56238) 2023-11-24T06:16:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.775+0000 7f0fe9596640 1 -- 172.21.15.5:0/1825376965 learned_addr learned my addr 172.21.15.5:0/1825376965 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.775+0000 7f0fe9d97640 1 --2- 172.21.15.5:0/1825376965 >> v2:172.21.15.5:3300/0 conn(0x7f0fe41460b0 0x7f0fe40b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:51.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.775+0000 7f0fea598640 1 --2- 172.21.15.5:0/1825376965 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fe40b3780 0x7f0fe415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:51.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.775+0000 7f0fe8d95640 1 -- 172.21.15.5:0/1825376965 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281712180 0 0) 0x7f0fe406cd50 con 0x7f0fe406b760 2023-11-24T06:16:51.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.775+0000 7f0fe8d95640 1 -- 172.21.15.5:0/1825376965 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fd0003520 con 0x7f0fe406b760 2023-11-24T06:16:51.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.775+0000 7f0fe9d97640 1 -- 172.21.15.5:0/1825376965 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fe40b3780 msgr2=0x7f0fe415d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:51.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.775+0000 7f0fe9d97640 1 --2- 172.21.15.5:0/1825376965 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fe40b3780 0x7f0fe415d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.775+0000 7f0fe9d97640 1 -- 172.21.15.5:0/1825376965 >> v1:172.21.15.5:6789/0 conn(0x7f0fe406b760 legacy=0x7f0fe40b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:51.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.775+0000 7f0fe9d97640 1 -- 172.21.15.5:0/1825376965 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe406bcb0 con 0x7f0fe41460b0 2023-11-24T06:16:51.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.775+0000 7f0fea598640 1 --2- 172.21.15.5:0/1825376965 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fe40b3780 0x7f0fe415d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:16:51.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.777+0000 7f0fe9d97640 1 --2- 172.21.15.5:0/1825376965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe41460b0 0x7f0fe40b4a00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00097a0 tx=0x7f0fe002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fd3fc61cd810a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.777+0000 7f0fe8d95640 1 -- 172.21.15.5:0/1825376965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0031780 con 0x7f0fe41460b0 2023-11-24T06:16:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.777+0000 7f0fe8d95640 1 -- 172.21.15.5:0/1825376965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe00318e0 con 0x7f0fe41460b0 2023-11-24T06:16:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.777+0000 7f0fe8d95640 1 -- 172.21.15.5:0/1825376965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0031c50 con 0x7f0fe41460b0 2023-11-24T06:16:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.777+0000 7f0febffb640 1 -- 172.21.15.5:0/1825376965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe41460b0 msgr2=0x7f0fe40b4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.777+0000 7f0febffb640 1 --2- 172.21.15.5:0/1825376965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe41460b0 0x7f0fe40b4a00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00097a0 tx=0x7f0fe002b820 comp rx=0 tx=0).stop 2023-11-24T06:16:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.778+0000 7f0febffb640 1 -- 172.21.15.5:0/1825376965 shutdown_connections 2023-11-24T06:16:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.778+0000 7f0febffb640 1 --2- 172.21.15.5:0/1825376965 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fe40b3780 0x7f0fe415d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.778+0000 7f0febffb640 1 --2- 172.21.15.5:0/1825376965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe41460b0 0x7f0fe40b4a00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.778+0000 7f0febffb640 1 -- 172.21.15.5:0/1825376965 >> 172.21.15.5:0/1825376965 conn(0x7f0fe40663d0 msgr2=0x7f0fe40667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.778+0000 7f0febffb640 1 -- 172.21.15.5:0/1825376965 shutdown_connections 2023-11-24T06:16:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.778+0000 7f0febffb640 1 -- 172.21.15.5:0/1825376965 wait complete. 2023-11-24T06:16:51.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.779+0000 7f0febffb640 1 Processor -- start 2023-11-24T06:16:51.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.780+0000 7f0febffb640 1 -- start start 2023-11-24T06:16:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0febffb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fe406b760 0x7f0fe4139900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0febffb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fe40b3780 0x7f0fe4139e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0febffb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe41460b0 0x7f0fe414eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0febffb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe414d430 con 0x7f0fe41460b0 2023-11-24T06:16:51.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0febffb640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe414cdb0 con 0x7f0fe406b760 2023-11-24T06:16:51.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0febffb640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe413c560 con 0x7f0fe40b3780 2023-11-24T06:16:51.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0fea598640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe41460b0 0x7f0fe414eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:51.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0fea598640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe41460b0 0x7f0fe414eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41010/0 (socket says 172.21.15.5:41010) 2023-11-24T06:16:51.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0fea598640 1 -- 172.21.15.5:0/1072455326 learned_addr learned my addr 172.21.15.5:0/1072455326 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:51.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0fe9d97640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fe406b760 0x7f0fe4139900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:51.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0fe9596640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fe40b3780 0x7f0fe4139e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:51.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0fea598640 1 -- 172.21.15.5:0/1072455326 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fe40b3780 msgr2=0x7f0fe4139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:51.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0fea598640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fe40b3780 0x7f0fe4139e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0fea598640 1 -- 172.21.15.5:0/1072455326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fe406b760 msgr2=0x7f0fe4139900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:51.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0fea598640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fe406b760 0x7f0fe4139900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:51.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0fea598640 1 -- 172.21.15.5:0/1072455326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe0009400 con 0x7f0fe41460b0 2023-11-24T06:16:51.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.781+0000 7f0fe9d97640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fe406b760 0x7f0fe4139900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:51.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.782+0000 7f0fea598640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe41460b0 0x7f0fe414eda0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc007e70 tx=0x7f0fdc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:51.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.782+0000 7f0fda7fc640 1 -- 172.21.15.5:0/1072455326 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc003b90 con 0x7f0fe41460b0 2023-11-24T06:16:51.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.782+0000 7f0fda7fc640 1 -- 172.21.15.5:0/1072455326 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc00f040 con 0x7f0fe41460b0 2023-11-24T06:16:51.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.782+0000 7f0fda7fc640 1 -- 172.21.15.5:0/1072455326 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc00de80 con 0x7f0fe41460b0 2023-11-24T06:16:51.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.782+0000 7f0febffb640 1 -- 172.21.15.5:0/1072455326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe414d050 con 0x7f0fe41460b0 2023-11-24T06:16:51.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.782+0000 7f0febffb640 1 -- 172.21.15.5:0/1072455326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe414f2e0 con 0x7f0fe41460b0 2023-11-24T06:16:51.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.783+0000 7f0febffb640 1 -- 172.21.15.5:0/1072455326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe414d570 con 0x7f0fe41460b0 2023-11-24T06:16:51.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.784+0000 7f0fda7fc640 1 -- 172.21.15.5:0/1072455326 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0fdc004030 con 0x7f0fe41460b0 2023-11-24T06:16:51.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.785+0000 7f0fda7fc640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0fc806ae40 0x7f0fc806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:51.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.785+0000 7f0fda7fc640 1 -- 172.21.15.5:0/1072455326 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0fdc089a00 con 0x7f0fe41460b0 2023-11-24T06:16:51.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.785+0000 7f0fe9d97640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0fc806ae40 0x7f0fc806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:51.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.786+0000 7f0fe9d97640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0fc806ae40 0x7f0fc806d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0036000 tx=0x7f0fe003d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.790+0000 7f0fda7fc640 1 -- 172.21.15.5:0/1072455326 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0fe414d570 con 0x7f0fe41460b0 2023-11-24T06:16:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.996+0000 7f0febffb640 1 -- 172.21.15.5:0/1072455326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0fe414d570 con 0x7f0fe41460b0 2023-11-24T06:16:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.997+0000 7f0fda7fc640 1 -- 172.21.15.5:0/1072455326 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0fe414d570 con 0x7f0fe41460b0 2023-11-24T06:16:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2023-11-24T06:16:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.999+0000 7f0febffb640 1 -- 172.21.15.5:0/1072455326 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0fc806ae40 msgr2=0x7f0fc806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.999+0000 7f0febffb640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0fc806ae40 0x7f0fc806d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0036000 tx=0x7f0fe003d040 comp rx=0 tx=0).stop 2023-11-24T06:16:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.999+0000 7f0febffb640 1 -- 172.21.15.5:0/1072455326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe41460b0 msgr2=0x7f0fe414eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.999+0000 7f0febffb640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe41460b0 0x7f0fe414eda0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc007e70 tx=0x7f0fdc00d4a0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.999+0000 7f0febffb640 1 -- 172.21.15.5:0/1072455326 shutdown_connections 2023-11-24T06:16:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.999+0000 7f0febffb640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0fc806ae40 0x7f0fc806d300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.999+0000 7f0febffb640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0fe40b3780 0x7f0fe4139e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.999+0000 7f0febffb640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fe41460b0 0x7f0fe414eda0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.999+0000 7f0febffb640 1 --2- 172.21.15.5:0/1072455326 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0fe406b760 0x7f0fe4139900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:51.999+0000 7f0febffb640 1 -- 172.21.15.5:0/1072455326 >> 172.21.15.5:0/1072455326 conn(0x7f0fe40663d0 msgr2=0x7f0fe4141870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.000+0000 7f0febffb640 1 -- 172.21.15.5:0/1072455326 shutdown_connections 2023-11-24T06:16:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.000+0000 7f0febffb640 1 -- 172.21.15.5:0/1072455326 wait complete. 2023-11-24T06:16:52.022 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:16:52.023 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:16:52.078 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:52.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.304+0000 7f6368e2d640 1 Processor -- start 2023-11-24T06:16:52.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.304+0000 7f6368e2d640 1 -- start start 2023-11-24T06:16:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.306+0000 7f6368e2d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f636415d520 0x7f63640cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.306+0000 7f6368e2d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63640b2aa0 0x7f63640ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.306+0000 7f6368e2d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63641613c0 con 0x7f63640b2f00 2023-11-24T06:16:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.306+0000 7f6368e2d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6364161f40 con 0x7f636415d520 2023-11-24T06:16:52.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.306+0000 7f6368e2d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63641616a0 con 0x7f63640b2aa0 2023-11-24T06:16:52.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.306+0000 7f6362d9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f63640b2f00 0x7f636415de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:56248/0 (socket says 172.21.15.5:56248) 2023-11-24T06:16:52.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.306+0000 7f6362d9d640 1 -- 172.21.15.5:0/2941572794 learned_addr learned my addr 172.21.15.5:0/2941572794 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:52.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.307+0000 7f6361d9b640 1 -- 172.21.15.5:0/2941572794 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978987956 0 0) 0x7f63641613c0 con 0x7f63640b2f00 2023-11-24T06:16:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.307+0000 7f635bfff640 1 --2- 172.21.15.5:0/2941572794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63640b2aa0 0x7f63640ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.307+0000 7f6361d9b640 1 -- 172.21.15.5:0/2941572794 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6350003520 con 0x7f63640b2f00 2023-11-24T06:16:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.307+0000 7f636259c640 1 --2- 172.21.15.5:0/2941572794 >> v2:172.21.15.6:3300/0 conn(0x7f636415d520 0x7f63640cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.307+0000 7f6361d9b640 1 -- 172.21.15.5:0/2941572794 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3052402756 0 0) 0x7f6350003520 con 0x7f63640b2f00 2023-11-24T06:16:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.307+0000 7f6361d9b640 1 -- 172.21.15.5:0/2941572794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63640b2aa0 msgr2=0x7f63640ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.307+0000 7f6361d9b640 1 --2- 172.21.15.5:0/2941572794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63640b2aa0 0x7f63640ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.307+0000 7f6361d9b640 1 -- 172.21.15.5:0/2941572794 >> v2:172.21.15.6:3300/0 conn(0x7f636415d520 msgr2=0x7f63640cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.307+0000 7f6361d9b640 1 --2- 172.21.15.5:0/2941572794 >> v2:172.21.15.6:3300/0 conn(0x7f636415d520 0x7f63640cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.307+0000 7f6361d9b640 1 -- 172.21.15.5:0/2941572794 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6364161980 con 0x7f63640b2f00 2023-11-24T06:16:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.307+0000 7f6361d9b640 1 -- 172.21.15.5:0/2941572794 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f635c0019f0 con 0x7f63640b2f00 2023-11-24T06:16:52.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.307+0000 7f6361d9b640 1 -- 172.21.15.5:0/2941572794 >> v1:172.21.15.5:6789/0 conn(0x7f63640b2f00 legacy=0x7f636415de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:52.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.307+0000 7f6361d9b640 1 --2- 172.21.15.5:0/2941572794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6350003970 0x7f6350003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.308+0000 7f6361d9b640 1 --2- 172.21.15.5:0/2941572794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63500047c0 0x7f6350004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.308+0000 7f6361d9b640 1 --2- 172.21.15.5:0/2941572794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6350005180 0x7f635000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.308+0000 7f6361d9b640 1 -- 172.21.15.5:0/2941572794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6364161f40 con 0x7f63500047c0 2023-11-24T06:16:52.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.308+0000 7f6361d9b640 1 -- 172.21.15.5:0/2941572794 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63641616a0 con 0x7f6350003970 2023-11-24T06:16:52.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.308+0000 7f6361d9b640 1 -- 172.21.15.5:0/2941572794 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635000bff0 con 0x7f6350005180 2023-11-24T06:16:52.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.308+0000 7f635bfff640 1 --2- 172.21.15.5:0/2941572794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63640b2aa0 0x7f63640ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:16:52.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.308+0000 7f635bfff640 1 --2- 172.21.15.5:0/2941572794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63500047c0 0x7f6350004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.308+0000 7f6362d9d640 1 --2- 172.21.15.5:0/2941572794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6350005180 0x7f635000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.308+0000 7f636259c640 1 --2- 172.21.15.5:0/2941572794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6350003970 0x7f6350003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.308+0000 7f635bfff640 1 -- 172.21.15.5:0/2941572794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6350005180 msgr2=0x7f635000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:52.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.308+0000 7f635bfff640 1 --2- 172.21.15.5:0/2941572794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6350005180 0x7f635000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.308+0000 7f635bfff640 1 -- 172.21.15.5:0/2941572794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6350003970 msgr2=0x7f6350003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:52.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.308+0000 7f635bfff640 1 --2- 172.21.15.5:0/2941572794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6350003970 0x7f6350003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.308+0000 7f635bfff640 1 -- 172.21.15.5:0/2941572794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6350003520 con 0x7f63500047c0 2023-11-24T06:16:52.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.310+0000 7f635bfff640 1 --2- 172.21.15.5:0/2941572794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63500047c0 0x7f6350004c40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f634c006fd0 tx=0x7f63500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe16a7e70a8dc9a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:52.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.310+0000 7f6361d9b640 1 -- 172.21.15.5:0/2941572794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c033600 con 0x7f63500047c0 2023-11-24T06:16:52.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.310+0000 7f6361d9b640 1 -- 172.21.15.5:0/2941572794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634c02f050 con 0x7f63500047c0 2023-11-24T06:16:52.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.310+0000 7f6361d9b640 1 -- 172.21.15.5:0/2941572794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c033980 con 0x7f63500047c0 2023-11-24T06:16:52.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.311+0000 7f6368e2d640 1 -- 172.21.15.5:0/2941572794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63500047c0 msgr2=0x7f6350004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:52.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.311+0000 7f6368e2d640 1 --2- 172.21.15.5:0/2941572794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63500047c0 0x7f6350004c40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f634c006fd0 tx=0x7f63500057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.311+0000 7f6362d9d640 1 -- 172.21.15.5:0/2941572794 reap_dead start 2023-11-24T06:16:52.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.311+0000 7f6368e2d640 1 -- 172.21.15.5:0/2941572794 shutdown_connections 2023-11-24T06:16:52.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.311+0000 7f6368e2d640 1 -- 172.21.15.5:0/2941572794 >> 172.21.15.5:0/2941572794 conn(0x7f63640b23a0 msgr2=0x7f63640cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:52.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.311+0000 7f6368e2d640 1 -- 172.21.15.5:0/2941572794 shutdown_connections 2023-11-24T06:16:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.311+0000 7f6368e2d640 1 -- 172.21.15.5:0/2941572794 wait complete. 2023-11-24T06:16:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.313+0000 7f6368e2d640 1 Processor -- start 2023-11-24T06:16:52.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.313+0000 7f6368e2d640 1 -- start start 2023-11-24T06:16:52.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.314+0000 7f6368e2d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63500047c0 0x7f636415eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.314+0000 7f6368e2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6364152720 0x7f636415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.314+0000 7f6368e2d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63641f6530 0x7f63641f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.314+0000 7f6368e2d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63641564e0 con 0x7f6364152720 2023-11-24T06:16:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.314+0000 7f6368e2d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63641f6ef0 con 0x7f63641f6530 2023-11-24T06:16:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.314+0000 7f6368e2d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63640ad920 con 0x7f63500047c0 2023-11-24T06:16:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.314+0000 7f635bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6364152720 0x7f636415dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.314+0000 7f635bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6364152720 0x7f636415dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41018/0 (socket says 172.21.15.5:41018) 2023-11-24T06:16:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.314+0000 7f635bfff640 1 -- 172.21.15.5:0/1081683514 learned_addr learned my addr 172.21.15.5:0/1081683514 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.315+0000 7f636259c640 1 --2- 172.21.15.5:0/1081683514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63500047c0 0x7f636415eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.315+0000 7f6362d9d640 1 --2- 172.21.15.5:0/1081683514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63641f6530 0x7f63641f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.315+0000 7f635bfff640 1 -- 172.21.15.5:0/1081683514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63500047c0 msgr2=0x7f636415eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:52.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.315+0000 7f635bfff640 1 --2- 172.21.15.5:0/1081683514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63500047c0 0x7f636415eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.315+0000 7f635bfff640 1 -- 172.21.15.5:0/1081683514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63641f6530 msgr2=0x7f63641f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:52.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.315+0000 7f635bfff640 1 --2- 172.21.15.5:0/1081683514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63641f6530 0x7f63641f69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.315+0000 7f635bfff640 1 -- 172.21.15.5:0/1081683514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63640adb90 con 0x7f6364152720 2023-11-24T06:16:52.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.315+0000 7f635bfff640 1 --2- 172.21.15.5:0/1081683514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6364152720 0x7f636415dd80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f634c02bdc0 tx=0x7f634c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:52.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.315+0000 7f635affd640 1 -- 172.21.15.5:0/1081683514 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c03d820 con 0x7f6364152720 2023-11-24T06:16:52.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.315+0000 7f635affd640 1 -- 172.21.15.5:0/1081683514 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634c03d980 con 0x7f6364152720 2023-11-24T06:16:52.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.316+0000 7f6368e2d640 1 -- 172.21.15.5:0/1081683514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63640ade70 con 0x7f6364152720 2023-11-24T06:16:52.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.316+0000 7f635affd640 1 -- 172.21.15.5:0/1081683514 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634c03dd30 con 0x7f6364152720 2023-11-24T06:16:52.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.316+0000 7f6368e2d640 1 -- 172.21.15.5:0/1081683514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6364152ba0 con 0x7f6364152720 2023-11-24T06:16:52.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.317+0000 7f6368e2d640 1 -- 172.21.15.5:0/1081683514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6364156650 con 0x7f6364152720 2023-11-24T06:16:52.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.318+0000 7f635affd640 1 -- 172.21.15.5:0/1081683514 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f634c046460 con 0x7f6364152720 2023-11-24T06:16:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.319+0000 7f635affd640 1 --2- 172.21.15.5:0/1081683514 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f634406ae70 0x7f634406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.319+0000 7f635affd640 1 -- 172.21.15.5:0/1081683514 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f634c0ab890 con 0x7f6364152720 2023-11-24T06:16:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.319+0000 7f636259c640 1 --2- 172.21.15.5:0/1081683514 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f634406ae70 0x7f634406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.320+0000 7f636259c640 1 --2- 172.21.15.5:0/1081683514 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f634406ae70 0x7f634406d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6354006fd0 tx=0x7f6354008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:52.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.324+0000 7f635affd640 1 -- 172.21.15.5:0/1081683514 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6364156650 con 0x7f6364152720 2023-11-24T06:16:52.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.522+0000 7f6368e2d640 1 -- 172.21.15.5:0/1081683514 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6364156620 con 0x7f6364152720 2023-11-24T06:16:52.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.524+0000 7f635affd640 1 -- 172.21.15.5:0/1081683514 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16658 (secure 0 0 0) 0x7f634c0768a0 con 0x7f6364152720 2023-11-24T06:16:52.527 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:52.528 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:16:33.958286+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.527+0000 7f6368e2d640 1 -- 172.21.15.5:0/1081683514 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f634406ae70 msgr2=0x7f634406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.527+0000 7f6368e2d640 1 --2- 172.21.15.5:0/1081683514 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f634406ae70 0x7f634406d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6354006fd0 tx=0x7f6354008040 comp rx=0 tx=0).stop 2023-11-24T06:16:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.527+0000 7f6368e2d640 1 -- 172.21.15.5:0/1081683514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6364152720 msgr2=0x7f636415dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.527+0000 7f6368e2d640 1 --2- 172.21.15.5:0/1081683514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6364152720 0x7f636415dd80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f634c02bdc0 tx=0x7f634c004730 comp rx=0 tx=0).stop 2023-11-24T06:16:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.528+0000 7f6368e2d640 1 -- 172.21.15.5:0/1081683514 shutdown_connections 2023-11-24T06:16:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.528+0000 7f6368e2d640 1 --2- 172.21.15.5:0/1081683514 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f634406ae70 0x7f634406d330 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.528+0000 7f6368e2d640 1 --2- 172.21.15.5:0/1081683514 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f63641f6530 0x7f63641f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.528+0000 7f6368e2d640 1 --2- 172.21.15.5:0/1081683514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6364152720 0x7f636415dd80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.528+0000 7f6368e2d640 1 --2- 172.21.15.5:0/1081683514 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f63500047c0 0x7f636415eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.528+0000 7f6368e2d640 1 -- 172.21.15.5:0/1081683514 >> 172.21.15.5:0/1081683514 conn(0x7f63640b23a0 msgr2=0x7f636415b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:52.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.528+0000 7f6368e2d640 1 -- 172.21.15.5:0/1081683514 shutdown_connections 2023-11-24T06:16:52.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.528+0000 7f6368e2d640 1 -- 172.21.15.5:0/1081683514 wait complete. 2023-11-24T06:16:52.550 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:52.551 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.768+0000 7fdf94885640 1 Processor -- start 2023-11-24T06:16:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.769+0000 7fdf94885640 1 -- start start 2023-11-24T06:16:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.770+0000 7fdf94885640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdf9015d3b0 0x7fdf900ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.770+0000 7fdf94885640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdf900d1d90 0x7fdf900b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.770+0000 7fdf94885640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf900c6bf0 con 0x7fdf900d0fa0 2023-11-24T06:16:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.770+0000 7fdf94885640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf900d1430 con 0x7fdf900d1d90 2023-11-24T06:16:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.770+0000 7fdf94885640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf900cedc0 con 0x7fdf9015d3b0 2023-11-24T06:16:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.771+0000 7fdf8ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdf900d1d90 0x7fdf900b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.771+0000 7fdf8ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdf900d1d90 0x7fdf900b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41030/0 (socket says 172.21.15.5:41030) 2023-11-24T06:16:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.771+0000 7fdf8ffff640 1 -- 172.21.15.5:0/465133019 learned_addr learned my addr 172.21.15.5:0/465133019 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.771+0000 7fdf8f7fe640 1 --2- 172.21.15.5:0/465133019 >> v2:172.21.15.6:3300/0 conn(0x7fdf9015d3b0 0x7fdf900ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.771+0000 7fdf8ffff640 1 -- 172.21.15.5:0/465133019 >> v2:172.21.15.6:3300/0 conn(0x7fdf9015d3b0 msgr2=0x7fdf900ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.771+0000 7fdf8ffff640 1 --2- 172.21.15.5:0/465133019 >> v2:172.21.15.6:3300/0 conn(0x7fdf9015d3b0 0x7fdf900ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.771+0000 7fdf8ffff640 1 -- 172.21.15.5:0/465133019 >> v1:172.21.15.6:6789/0 conn(0x7fdf900d0fa0 legacy=0x7fdf900d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.771+0000 7fdf8ffff640 1 -- 172.21.15.5:0/465133019 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf900c6e40 con 0x7fdf900d1d90 2023-11-24T06:16:52.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.771+0000 7fdf8f7fe640 1 --2- 172.21.15.5:0/465133019 >> v2:172.21.15.6:3300/0 conn(0x7fdf9015d3b0 0x7fdf900ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.773+0000 7fdf8ffff640 1 --2- 172.21.15.5:0/465133019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf900d1d90 0x7fdf900b3ca0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdf9008b2a0 tx=0x7fdf8802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2cbc74535f7d59 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.773+0000 7fdf8e7fc640 1 -- 172.21.15.5:0/465133019 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf880317e0 con 0x7fdf900d1d90 2023-11-24T06:16:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.773+0000 7fdf8e7fc640 1 -- 172.21.15.5:0/465133019 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf88031940 con 0x7fdf900d1d90 2023-11-24T06:16:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.773+0000 7fdf8e7fc640 1 -- 172.21.15.5:0/465133019 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88031cd0 con 0x7fdf900d1d90 2023-11-24T06:16:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.774+0000 7fdf94885640 1 -- 172.21.15.5:0/465133019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf900d1d90 msgr2=0x7fdf900b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.774+0000 7fdf94885640 1 --2- 172.21.15.5:0/465133019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf900d1d90 0x7fdf900b3ca0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdf9008b2a0 tx=0x7fdf8802b880 comp rx=0 tx=0).stop 2023-11-24T06:16:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.774+0000 7fdf94885640 1 -- 172.21.15.5:0/465133019 shutdown_connections 2023-11-24T06:16:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.774+0000 7fdf94885640 1 --2- 172.21.15.5:0/465133019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf900d1d90 0x7fdf900b3ca0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.774+0000 7fdf94885640 1 --2- 172.21.15.5:0/465133019 >> v2:172.21.15.6:3300/0 conn(0x7fdf9015d3b0 0x7fdf900ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.774+0000 7fdf94885640 1 -- 172.21.15.5:0/465133019 >> 172.21.15.5:0/465133019 conn(0x7fdf900d30a0 msgr2=0x7fdf900d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.774+0000 7fdf94885640 1 -- 172.21.15.5:0/465133019 shutdown_connections 2023-11-24T06:16:52.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.774+0000 7fdf94885640 1 -- 172.21.15.5:0/465133019 wait complete. 2023-11-24T06:16:52.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.776+0000 7fdf94885640 1 Processor -- start 2023-11-24T06:16:52.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.776+0000 7fdf94885640 1 -- start start 2023-11-24T06:16:52.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf94885640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf900d0fa0 0x7fdf9015f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf94885640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf900d1d90 0x7fdf9015d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf94885640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9015d3b0 0x7fdf9015de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf94885640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9015f870 con 0x7fdf9015d3b0 2023-11-24T06:16:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf94885640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf900b69d0 con 0x7fdf900d1d90 2023-11-24T06:16:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf94885640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf900b5e00 con 0x7fdf900d0fa0 2023-11-24T06:16:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf8ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9015d3b0 0x7fdf9015de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf8ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9015d3b0 0x7fdf9015de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41038/0 (socket says 172.21.15.5:41038) 2023-11-24T06:16:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf8ffff640 1 -- 172.21.15.5:0/753688766 learned_addr learned my addr 172.21.15.5:0/753688766 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf8f7fe640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf900d0fa0 0x7fdf9015f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf8effd640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf900d1d90 0x7fdf9015d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf8ffff640 1 -- 172.21.15.5:0/753688766 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf900d0fa0 msgr2=0x7fdf9015f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:52.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf8ffff640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf900d0fa0 0x7fdf9015f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf8ffff640 1 -- 172.21.15.5:0/753688766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf900d1d90 msgr2=0x7fdf9015d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf8ffff640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf900d1d90 0x7fdf9015d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf8ffff640 1 -- 172.21.15.5:0/753688766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf880094d0 con 0x7fdf9015d3b0 2023-11-24T06:16:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.777+0000 7fdf8f7fe640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf900d0fa0 0x7fdf9015f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.778+0000 7fdf8effd640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf900d1d90 0x7fdf9015d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:52.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.778+0000 7fdf8ffff640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9015d3b0 0x7fdf9015de50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdf880090b0 tx=0x7fdf880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.778+0000 7fdf87fff640 1 -- 172.21.15.5:0/753688766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88032940 con 0x7fdf9015d3b0 2023-11-24T06:16:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.778+0000 7fdf87fff640 1 -- 172.21.15.5:0/753688766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf88032ac0 con 0x7fdf9015d3b0 2023-11-24T06:16:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.778+0000 7fdf87fff640 1 -- 172.21.15.5:0/753688766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88032e70 con 0x7fdf9015d3b0 2023-11-24T06:16:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.778+0000 7fdf94885640 1 -- 172.21.15.5:0/753688766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf90156bc0 con 0x7fdf9015d3b0 2023-11-24T06:16:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.778+0000 7fdf94885640 1 -- 172.21.15.5:0/753688766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf900b6380 con 0x7fdf9015d3b0 2023-11-24T06:16:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.779+0000 7fdf94885640 1 -- 172.21.15.5:0/753688766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf900b6b40 con 0x7fdf9015d3b0 2023-11-24T06:16:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.780+0000 7fdf87fff640 1 -- 172.21.15.5:0/753688766 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdf8804e450 con 0x7fdf9015d3b0 2023-11-24T06:16:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.781+0000 7fdf87fff640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf7006ae40 0x7fdf7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.781+0000 7fdf87fff640 1 -- 172.21.15.5:0/753688766 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fdf880ab930 con 0x7fdf9015d3b0 2023-11-24T06:16:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.782+0000 7fdf8f7fe640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf7006ae40 0x7fdf7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:52.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.782+0000 7fdf8f7fe640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf7006ae40 0x7fdf7006d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdf80007f40 tx=0x7fdf80007490 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.786+0000 7fdf87fff640 1 -- 172.21.15.5:0/753688766 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf88076b00 con 0x7fdf9015d3b0 2023-11-24T06:16:52.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.979+0000 7fdf94885640 1 -- 172.21.15.5:0/753688766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf9004e840 con 0x7fdf9015d3b0 2023-11-24T06:16:52.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.980+0000 7fdf87fff640 1 -- 172.21.15.5:0/753688766 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf8807a120 con 0x7fdf9015d3b0 2023-11-24T06:16:52.984 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:52.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.982+0000 7fdf94885640 1 -- 172.21.15.5:0/753688766 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf7006ae40 msgr2=0x7fdf7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:52.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.982+0000 7fdf94885640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf7006ae40 0x7fdf7006d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdf80007f40 tx=0x7fdf80007490 comp rx=0 tx=0).stop 2023-11-24T06:16:52.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.982+0000 7fdf94885640 1 -- 172.21.15.5:0/753688766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9015d3b0 msgr2=0x7fdf9015de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:52.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.982+0000 7fdf94885640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9015d3b0 0x7fdf9015de50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdf880090b0 tx=0x7fdf880046c0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.983+0000 7fdf94885640 1 -- 172.21.15.5:0/753688766 shutdown_connections 2023-11-24T06:16:52.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.983+0000 7fdf94885640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf7006ae40 0x7fdf7006d300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.983+0000 7fdf94885640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf9015d3b0 0x7fdf9015de50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.983+0000 7fdf94885640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf900d1d90 0x7fdf9015d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.983+0000 7fdf94885640 1 --2- 172.21.15.5:0/753688766 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf900d0fa0 0x7fdf9015f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:52.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.983+0000 7fdf94885640 1 -- 172.21.15.5:0/753688766 >> 172.21.15.5:0/753688766 conn(0x7fdf900d30a0 msgr2=0x7fdf900c85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:52.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.983+0000 7fdf94885640 1 -- 172.21.15.5:0/753688766 shutdown_connections 2023-11-24T06:16:52.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:52.983+0000 7fdf94885640 1 -- 172.21.15.5:0/753688766 wait complete. 2023-11-24T06:16:53.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:53.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.233+0000 7f82bbfd5640 1 Processor -- start 2023-11-24T06:16:53.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.233+0000 7f82bbfd5640 1 -- start start 2023-11-24T06:16:53.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.235+0000 7f82bbfd5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f82b40ae270 0x7f82b40abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:53.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.235+0000 7f82bbfd5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b415b730 con 0x7f82b415b350 2023-11-24T06:16:53.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.235+0000 7f82bbfd5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b40ac740 con 0x7f82b409cb30 2023-11-24T06:16:53.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.235+0000 7f82bbfd5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b40ac460 con 0x7f82b40ae270 2023-11-24T06:16:53.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.235+0000 7f82b9d71640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f82b415b350 0x7f82b40b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46242/0 (socket says 172.21.15.5:46242) 2023-11-24T06:16:53.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.235+0000 7f82b9d71640 1 -- 172.21.15.5:0/54072362 learned_addr learned my addr 172.21.15.5:0/54072362 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:53.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.235+0000 7f82b8d6f640 1 -- 172.21.15.5:0/54072362 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650151311 0 0) 0x7f82b0001110 con 0x7f82b415b350 2023-11-24T06:16:53.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.236+0000 7f82ba572640 1 --2- 172.21.15.5:0/54072362 >> v2:172.21.15.6:3300/0 conn(0x7f82b40ae270 0x7f82b40abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:53.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.236+0000 7f82b8d6f640 1 -- 172.21.15.5:0/54072362 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82a0003520 con 0x7f82b415b350 2023-11-24T06:16:53.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.236+0000 7f82b8d6f640 1 -- 172.21.15.5:0/54072362 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2624455963 0 0) 0x7f82a0003520 con 0x7f82b415b350 2023-11-24T06:16:53.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.236+0000 7f82b8d6f640 1 -- 172.21.15.5:0/54072362 >> v2:172.21.15.6:3300/0 conn(0x7f82b40ae270 msgr2=0x7f82b40abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:53.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.236+0000 7f82b8d6f640 1 --2- 172.21.15.5:0/54072362 >> v2:172.21.15.6:3300/0 conn(0x7f82b40ae270 0x7f82b40abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.236+0000 7f82b8d6f640 1 -- 172.21.15.5:0/54072362 >> v1:172.21.15.6:6789/0 conn(0x7f82b409cb30 legacy=0x7f82b40adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:53.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.236+0000 7f82b8d6f640 1 -- 172.21.15.5:0/54072362 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b40b2a50 con 0x7f82b415b350 2023-11-24T06:16:53.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.236+0000 7f82b8d6f640 1 -- 172.21.15.5:0/54072362 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f82b0001800 con 0x7f82b415b350 2023-11-24T06:16:53.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.236+0000 7f82b8d6f640 1 -- 172.21.15.5:0/54072362 >> v1:172.21.15.5:6789/0 conn(0x7f82b415b350 legacy=0x7f82b40b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:53.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.236+0000 7f82b8d6f640 1 --2- 172.21.15.5:0/54072362 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82a0003970 0x7f82a0003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.237+0000 7f82b8d6f640 1 --2- 172.21.15.5:0/54072362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a00047c0 0x7f82a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.237+0000 7f82b8d6f640 1 --2- 172.21.15.5:0/54072362 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82a0005180 0x7f82a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.237+0000 7f82b8d6f640 1 -- 172.21.15.5:0/54072362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b40ac460 con 0x7f82a00047c0 2023-11-24T06:16:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.237+0000 7f82b8d6f640 1 -- 172.21.15.5:0/54072362 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a000c0b0 con 0x7f82a0005180 2023-11-24T06:16:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.237+0000 7f82b8d6f640 1 -- 172.21.15.5:0/54072362 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a000c270 con 0x7f82a0003970 2023-11-24T06:16:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.237+0000 7f82b9570640 1 --2- 172.21.15.5:0/54072362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a00047c0 0x7f82a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.237+0000 7f82b9d71640 1 --2- 172.21.15.5:0/54072362 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82a0003970 0x7f82a0003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.237+0000 7f82b9570640 1 -- 172.21.15.5:0/54072362 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82a0003970 msgr2=0x7f82a0003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.237+0000 7f82ba572640 1 --2- 172.21.15.5:0/54072362 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82a0005180 0x7f82a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.237+0000 7f82b9570640 1 --2- 172.21.15.5:0/54072362 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82a0003970 0x7f82a0003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.237+0000 7f82b9570640 1 -- 172.21.15.5:0/54072362 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82a0005180 msgr2=0x7f82a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:53.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.237+0000 7f82b9570640 1 --2- 172.21.15.5:0/54072362 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82a0005180 0x7f82a000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.237+0000 7f82b9570640 1 -- 172.21.15.5:0/54072362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a0003520 con 0x7f82a00047c0 2023-11-24T06:16:53.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.238+0000 7f82ba572640 1 --2- 172.21.15.5:0/54072362 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82a0005180 0x7f82a000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:53.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.238+0000 7f82ba572640 1 -- 172.21.15.5:0/54072362 reap_dead start 2023-11-24T06:16:53.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.238+0000 7f82b9d71640 1 --2- 172.21.15.5:0/54072362 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82a0003970 0x7f82a0003d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:53.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.239+0000 7f82b9570640 1 --2- 172.21.15.5:0/54072362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a00047c0 0x7f82a0004c40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f82a4001650 tx=0x7f82a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9ea4ead84004222 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:53.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.239+0000 7f82b8d6f640 1 -- 172.21.15.5:0/54072362 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4005270 con 0x7f82a00047c0 2023-11-24T06:16:53.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.239+0000 7f82b8d6f640 1 -- 172.21.15.5:0/54072362 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a402e070 con 0x7f82a00047c0 2023-11-24T06:16:53.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.239+0000 7f82b8d6f640 1 -- 172.21.15.5:0/54072362 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a4003610 con 0x7f82a00047c0 2023-11-24T06:16:53.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.239+0000 7f82bbfd5640 1 -- 172.21.15.5:0/54072362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a00047c0 msgr2=0x7f82a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:53.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.239+0000 7f82bbfd5640 1 --2- 172.21.15.5:0/54072362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a00047c0 0x7f82a0004c40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f82a4001650 tx=0x7f82a00057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.240+0000 7f82bbfd5640 1 -- 172.21.15.5:0/54072362 shutdown_connections 2023-11-24T06:16:53.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.240+0000 7f82bbfd5640 1 --2- 172.21.15.5:0/54072362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a00047c0 0x7f82a0004c40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.240+0000 7f82bbfd5640 1 -- 172.21.15.5:0/54072362 >> 172.21.15.5:0/54072362 conn(0x7f82b40b0ba0 msgr2=0x7f82b40b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:53.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.240+0000 7f82bbfd5640 1 -- 172.21.15.5:0/54072362 shutdown_connections 2023-11-24T06:16:53.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.240+0000 7f82bbfd5640 1 -- 172.21.15.5:0/54072362 wait complete. 2023-11-24T06:16:53.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.242+0000 7f82bbfd5640 1 Processor -- start 2023-11-24T06:16:53.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.242+0000 7f82bbfd5640 1 -- start start 2023-11-24T06:16:53.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.243+0000 7f82bbfd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a00047c0 0x7f82b415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:53.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.243+0000 7f82bbfd5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82b409bbf0 0x7f82b414b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:53.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.243+0000 7f82bbfd5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82b415db60 0x7f82b415dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:53.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.243+0000 7f82bbfd5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b415e530 con 0x7f82a00047c0 2023-11-24T06:16:53.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.243+0000 7f82bbfd5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b415c8c0 con 0x7f82b415db60 2023-11-24T06:16:53.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.243+0000 7f82bbfd5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b415cba0 con 0x7f82b409bbf0 2023-11-24T06:16:53.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.243+0000 7f82b9d71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a00047c0 0x7f82b415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:53.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.244+0000 7f82b9d71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a00047c0 0x7f82b415c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42816/0 (socket says 172.21.15.5:42816) 2023-11-24T06:16:53.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.244+0000 7f82b9d71640 1 -- 172.21.15.5:0/3673074451 learned_addr learned my addr 172.21.15.5:0/3673074451 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.244+0000 7f82b9570640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82b409bbf0 0x7f82b414b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.244+0000 7f82ba572640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82b415db60 0x7f82b415dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.244+0000 7f82b9d71640 1 -- 172.21.15.5:0/3673074451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82b409bbf0 msgr2=0x7f82b414b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.244+0000 7f82b9d71640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82b409bbf0 0x7f82b414b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.244+0000 7f82b9d71640 1 -- 172.21.15.5:0/3673074451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82b415db60 msgr2=0x7f82b415dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.244+0000 7f82b9d71640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82b415db60 0x7f82b415dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.244+0000 7f82b9d71640 1 -- 172.21.15.5:0/3673074451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b41577f0 con 0x7f82a00047c0 2023-11-24T06:16:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.244+0000 7f82b9570640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82b409bbf0 0x7f82b414b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.244+0000 7f82b9d71640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a00047c0 0x7f82b415c2e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f82b0000c00 tx=0x7f82b00083b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.245+0000 7f82aa7fc640 1 -- 172.21.15.5:0/3673074451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b0007150 con 0x7f82a00047c0 2023-11-24T06:16:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.245+0000 7f82aa7fc640 1 -- 172.21.15.5:0/3673074451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b00072b0 con 0x7f82a00047c0 2023-11-24T06:16:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.245+0000 7f82aa7fc640 1 -- 172.21.15.5:0/3673074451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b0007640 con 0x7f82a00047c0 2023-11-24T06:16:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.245+0000 7f82bbfd5640 1 -- 172.21.15.5:0/3673074451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b4157ad0 con 0x7f82a00047c0 2023-11-24T06:16:53.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.245+0000 7f82bbfd5640 1 -- 172.21.15.5:0/3673074451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b414b620 con 0x7f82a00047c0 2023-11-24T06:16:53.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.246+0000 7f82bbfd5640 1 -- 172.21.15.5:0/3673074451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b4157650 con 0x7f82a00047c0 2023-11-24T06:16:53.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.247+0000 7f82aa7fc640 1 -- 172.21.15.5:0/3673074451 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f82b00077a0 con 0x7f82a00047c0 2023-11-24T06:16:53.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.248+0000 7f82aa7fc640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f829806ae70 0x7f829806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:53.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.249+0000 7f82aa7fc640 1 -- 172.21.15.5:0/3673074451 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f82b0089760 con 0x7f82a00047c0 2023-11-24T06:16:53.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.249+0000 7f82b9570640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f829806ae70 0x7f829806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:53.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.249+0000 7f82b9570640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f829806ae70 0x7f829806d330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f82a4033000 tx=0x7f82a4009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:53.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.253+0000 7f82aa7fc640 1 -- 172.21.15.5:0/3673074451 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f82b4157650 con 0x7f82a00047c0 2023-11-24T06:16:53.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.449+0000 7f82bbfd5640 1 -- 172.21.15.5:0/3673074451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f82b415b730 con 0x7f82a00047c0 2023-11-24T06:16:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.450+0000 7f82aa7fc640 1 -- 172.21.15.5:0/3673074451 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f82b415b730 con 0x7f82a00047c0 2023-11-24T06:16:53.453 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.452+0000 7f82bbfd5640 1 -- 172.21.15.5:0/3673074451 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f829806ae70 msgr2=0x7f829806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.452+0000 7f82bbfd5640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f829806ae70 0x7f829806d330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f82a4033000 tx=0x7f82a4009030 comp rx=0 tx=0).stop 2023-11-24T06:16:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.452+0000 7f82bbfd5640 1 -- 172.21.15.5:0/3673074451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a00047c0 msgr2=0x7f82b415c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.452+0000 7f82bbfd5640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a00047c0 0x7f82b415c2e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f82b0000c00 tx=0x7f82b00083b0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.453+0000 7f82bbfd5640 1 -- 172.21.15.5:0/3673074451 shutdown_connections 2023-11-24T06:16:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.453+0000 7f82bbfd5640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f829806ae70 0x7f829806d330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.453+0000 7f82bbfd5640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82b409bbf0 0x7f82b414b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.453+0000 7f82bbfd5640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82b415db60 0x7f82b415dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.453+0000 7f82bbfd5640 1 --2- 172.21.15.5:0/3673074451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82a00047c0 0x7f82b415c2e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.453+0000 7f82bbfd5640 1 -- 172.21.15.5:0/3673074451 >> 172.21.15.5:0/3673074451 conn(0x7f82b40b0ba0 msgr2=0x7f82b40b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.453+0000 7f82bbfd5640 1 -- 172.21.15.5:0/3673074451 shutdown_connections 2023-11-24T06:16:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.453+0000 7f82bbfd5640 1 -- 172.21.15.5:0/3673074451 wait complete. 2023-11-24T06:16:53.473 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:53.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.697+0000 7f2463fff640 1 Processor -- start 2023-11-24T06:16:53.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.698+0000 7f2463fff640 1 -- start start 2023-11-24T06:16:53.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.699+0000 7f2463fff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f24641439e0 0x7f24640b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:53.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.699+0000 7f2463fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f24640b43f0 0x7f246406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:53.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.699+0000 7f2463fff640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24640b4dc0 con 0x7f246406a440 2023-11-24T06:16:53.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.699+0000 7f2463fff640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f246406ae30 con 0x7f24640b43f0 2023-11-24T06:16:53.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.699+0000 7f2463fff640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2464143dc0 con 0x7f24641439e0 2023-11-24T06:16:53.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.700+0000 7f24637fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f24640b43f0 0x7f246406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:53.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.700+0000 7f24637fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f24640b43f0 0x7f246406a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42826/0 (socket says 172.21.15.5:42826) 2023-11-24T06:16:53.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.700+0000 7f24637fe640 1 -- 172.21.15.5:0/3426293133 learned_addr learned my addr 172.21.15.5:0/3426293133 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:53.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.700+0000 7f2462ffd640 1 --2- 172.21.15.5:0/3426293133 >> v2:172.21.15.6:3300/0 conn(0x7f24641439e0 0x7f24640b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:53.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.700+0000 7f24637fe640 1 -- 172.21.15.5:0/3426293133 >> v2:172.21.15.6:3300/0 conn(0x7f24641439e0 msgr2=0x7f24640b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:53.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.700+0000 7f24637fe640 1 --2- 172.21.15.5:0/3426293133 >> v2:172.21.15.6:3300/0 conn(0x7f24641439e0 0x7f24640b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.700+0000 7f24637fe640 1 -- 172.21.15.5:0/3426293133 >> v1:172.21.15.6:6789/0 conn(0x7f246406a440 legacy=0x7f246406d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:53.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.700+0000 7f24637fe640 1 -- 172.21.15.5:0/3426293133 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2464161350 con 0x7f24640b43f0 2023-11-24T06:16:53.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.700+0000 7f2462ffd640 1 --2- 172.21.15.5:0/3426293133 >> v2:172.21.15.6:3300/0 conn(0x7f24641439e0 0x7f24640b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:53.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.702+0000 7f24637fe640 1 --2- 172.21.15.5:0/3426293133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24640b43f0 0x7f246406a8f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f246406c4c0 tx=0x7f245c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1449ea67b32f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:53.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.702+0000 7f2461ffb640 1 -- 172.21.15.5:0/3426293133 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c031870 con 0x7f24640b43f0 2023-11-24T06:16:53.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.702+0000 7f2461ffb640 1 -- 172.21.15.5:0/3426293133 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245c0319d0 con 0x7f24640b43f0 2023-11-24T06:16:53.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.702+0000 7f2461ffb640 1 -- 172.21.15.5:0/3426293133 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c031d60 con 0x7f24640b43f0 2023-11-24T06:16:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.703+0000 7f2463fff640 1 -- 172.21.15.5:0/3426293133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24640b43f0 msgr2=0x7f246406a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.703+0000 7f2463fff640 1 --2- 172.21.15.5:0/3426293133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24640b43f0 0x7f246406a8f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f246406c4c0 tx=0x7f245c02b910 comp rx=0 tx=0).stop 2023-11-24T06:16:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.703+0000 7f2463fff640 1 -- 172.21.15.5:0/3426293133 shutdown_connections 2023-11-24T06:16:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.703+0000 7f2463fff640 1 --2- 172.21.15.5:0/3426293133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24640b43f0 0x7f246406a8f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.703+0000 7f2463fff640 1 --2- 172.21.15.5:0/3426293133 >> v2:172.21.15.6:3300/0 conn(0x7f24641439e0 0x7f24640b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.703+0000 7f2463fff640 1 -- 172.21.15.5:0/3426293133 >> 172.21.15.5:0/3426293133 conn(0x7f246406bbb0 msgr2=0x7f24640b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:53.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.703+0000 7f2463fff640 1 -- 172.21.15.5:0/3426293133 shutdown_connections 2023-11-24T06:16:53.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.703+0000 7f2463fff640 1 -- 172.21.15.5:0/3426293133 wait complete. 2023-11-24T06:16:53.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.705+0000 7f2463fff640 1 Processor -- start 2023-11-24T06:16:53.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.705+0000 7f2463fff640 1 -- start start 2023-11-24T06:16:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.706+0000 7f2463fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246406a440 0x7f24641f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:53.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.706+0000 7f2463fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24640b43f0 0x7f246415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.706+0000 7f2463fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24641439e0 0x7f2464141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.706+0000 7f2463fff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24641f34d0 con 0x7f246406a440 2023-11-24T06:16:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.706+0000 7f2463fff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246415e870 con 0x7f24641439e0 2023-11-24T06:16:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.706+0000 7f2463fff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246415e2b0 con 0x7f24640b43f0 2023-11-24T06:16:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.706+0000 7f2462ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246406a440 0x7f24641f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.706+0000 7f2462ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246406a440 0x7f24641f38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42838/0 (socket says 172.21.15.5:42838) 2023-11-24T06:16:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.706+0000 7f2462ffd640 1 -- 172.21.15.5:0/1444555447 learned_addr learned my addr 172.21.15.5:0/1444555447 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.707+0000 7f24627fc640 1 --2- 172.21.15.5:0/1444555447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24640b43f0 0x7f246415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.707+0000 7f24637fe640 1 --2- 172.21.15.5:0/1444555447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24641439e0 0x7f2464141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:53.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.707+0000 7f2462ffd640 1 -- 172.21.15.5:0/1444555447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24640b43f0 msgr2=0x7f246415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.707+0000 7f2462ffd640 1 --2- 172.21.15.5:0/1444555447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24640b43f0 0x7f246415d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.707+0000 7f2462ffd640 1 -- 172.21.15.5:0/1444555447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24641439e0 msgr2=0x7f2464141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.707+0000 7f2462ffd640 1 --2- 172.21.15.5:0/1444555447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24641439e0 0x7f2464141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.707+0000 7f2462ffd640 1 -- 172.21.15.5:0/1444555447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245c0094d0 con 0x7f246406a440 2023-11-24T06:16:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.707+0000 7f2462ffd640 1 --2- 172.21.15.5:0/1444555447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246406a440 0x7f24641f38f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f245400ce00 tx=0x7f245400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.707+0000 7f244b7fe640 1 -- 172.21.15.5:0/1444555447 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245400f890 con 0x7f246406a440 2023-11-24T06:16:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.707+0000 7f244b7fe640 1 -- 172.21.15.5:0/1444555447 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245400f9f0 con 0x7f246406a440 2023-11-24T06:16:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.707+0000 7f244b7fe640 1 -- 172.21.15.5:0/1444555447 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245400fdc0 con 0x7f246406a440 2023-11-24T06:16:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.707+0000 7f2463fff640 1 -- 172.21.15.5:0/1444555447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f246415e590 con 0x7f246406a440 2023-11-24T06:16:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.708+0000 7f2463fff640 1 -- 172.21.15.5:0/1444555447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24641f3e30 con 0x7f246406a440 2023-11-24T06:16:53.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.709+0000 7f2463fff640 1 -- 172.21.15.5:0/1444555447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24641f2f50 con 0x7f246406a440 2023-11-24T06:16:53.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.710+0000 7f244b7fe640 1 -- 172.21.15.5:0/1444555447 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f245400b040 con 0x7f246406a440 2023-11-24T06:16:53.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.711+0000 7f244b7fe640 1 --2- 172.21.15.5:0/1444555447 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f245006ae40 0x7f245006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:53.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.711+0000 7f244b7fe640 1 -- 172.21.15.5:0/1444555447 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f245408a0e0 con 0x7f246406a440 2023-11-24T06:16:53.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.711+0000 7f24627fc640 1 --2- 172.21.15.5:0/1444555447 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f245006ae40 0x7f245006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:53.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.712+0000 7f24627fc640 1 --2- 172.21.15.5:0/1444555447 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f245006ae40 0x7f245006d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2458007fd0 tx=0x7f2458009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:53.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.716+0000 7f244b7fe640 1 -- 172.21.15.5:0/1444555447 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f24641f2f50 con 0x7f246406a440 2023-11-24T06:16:53.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.912+0000 7f2463fff640 1 -- 172.21.15.5:0/1444555447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f24641f2f50 con 0x7f246406a440 2023-11-24T06:16:53.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.913+0000 7f244b7fe640 1 -- 172.21.15.5:0/1444555447 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f24641f2f50 con 0x7f246406a440 2023-11-24T06:16:53.916 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.916+0000 7f2463fff640 1 -- 172.21.15.5:0/1444555447 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f245006ae40 msgr2=0x7f245006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.916+0000 7f2463fff640 1 --2- 172.21.15.5:0/1444555447 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f245006ae40 0x7f245006d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2458007fd0 tx=0x7f2458009040 comp rx=0 tx=0).stop 2023-11-24T06:16:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.916+0000 7f2463fff640 1 -- 172.21.15.5:0/1444555447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246406a440 msgr2=0x7f24641f38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.916+0000 7f2463fff640 1 --2- 172.21.15.5:0/1444555447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246406a440 0x7f24641f38f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f245400ce00 tx=0x7f245400a650 comp rx=0 tx=0).stop 2023-11-24T06:16:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.916+0000 7f2463fff640 1 -- 172.21.15.5:0/1444555447 shutdown_connections 2023-11-24T06:16:53.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.916+0000 7f2463fff640 1 --2- 172.21.15.5:0/1444555447 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f245006ae40 0x7f245006d300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.916+0000 7f2463fff640 1 --2- 172.21.15.5:0/1444555447 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f24640b43f0 0x7f246415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.916+0000 7f2463fff640 1 --2- 172.21.15.5:0/1444555447 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f24641439e0 0x7f2464141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.916+0000 7f2463fff640 1 --2- 172.21.15.5:0/1444555447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f246406a440 0x7f24641f38f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.916+0000 7f2463fff640 1 -- 172.21.15.5:0/1444555447 >> 172.21.15.5:0/1444555447 conn(0x7f246406bbb0 msgr2=0x7f246406f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.916+0000 7f2463fff640 1 -- 172.21.15.5:0/1444555447 shutdown_connections 2023-11-24T06:16:53.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:53.916+0000 7f2463fff640 1 -- 172.21.15.5:0/1444555447 wait complete. 2023-11-24T06:16:53.938 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:16:54.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.163+0000 7f0977fff640 1 Processor -- start 2023-11-24T06:16:54.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.164+0000 7f0977fff640 1 -- start start 2023-11-24T06:16:54.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.166+0000 7f0977fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f09780b2550 0x7f0978070550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.166+0000 7f0977fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0978161700 0x7f09780ac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.166+0000 7f0977fff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0978070a90 con 0x7f09780b2930 2023-11-24T06:16:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.166+0000 7f0977fff640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0978070cf0 con 0x7f09780b2550 2023-11-24T06:16:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.166+0000 7f0977fff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0978070fd0 con 0x7f0978161700 2023-11-24T06:16:54.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.166+0000 7f096ffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f09780b2930 0x7f0978160ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46248/0 (socket says 172.21.15.5:46248) 2023-11-24T06:16:54.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.166+0000 7f096ffff640 1 -- 172.21.15.5:0/4179135585 learned_addr learned my addr 172.21.15.5:0/4179135585 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:54.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.166+0000 7f0976ffd640 1 --2- 172.21.15.5:0/4179135585 >> v2:172.21.15.5:3300/0 conn(0x7f09780b2550 0x7f0978070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:54.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.167+0000 7f09767fc640 1 -- 172.21.15.5:0/4179135585 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243797435 0 0) 0x7f0978070a90 con 0x7f09780b2930 2023-11-24T06:16:54.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.167+0000 7f09767fc640 1 -- 172.21.15.5:0/4179135585 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0964003520 con 0x7f09780b2930 2023-11-24T06:16:54.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.167+0000 7f09777fe640 1 --2- 172.21.15.5:0/4179135585 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0978161700 0x7f09780ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:54.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.167+0000 7f0976ffd640 1 -- 172.21.15.5:0/4179135585 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0978161700 msgr2=0x7f09780ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:54.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.167+0000 7f0976ffd640 1 --2- 172.21.15.5:0/4179135585 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0978161700 0x7f09780ac900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.167+0000 7f0976ffd640 1 -- 172.21.15.5:0/4179135585 >> v1:172.21.15.5:6789/0 conn(0x7f09780b2930 legacy=0x7f0978160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:54.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.167+0000 7f0976ffd640 1 -- 172.21.15.5:0/4179135585 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09780712b0 con 0x7f09780b2550 2023-11-24T06:16:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.168+0000 7f0976ffd640 1 --2- 172.21.15.5:0/4179135585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09780b2550 0x7f0978070550 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f09680097a0 tx=0x7f096802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1488545db2f70c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.168+0000 7f09767fc640 1 -- 172.21.15.5:0/4179135585 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096802d040 con 0x7f09780b2550 2023-11-24T06:16:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.168+0000 7f09767fc640 1 -- 172.21.15.5:0/4179135585 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0968031810 con 0x7f09780b2550 2023-11-24T06:16:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.168+0000 7f09767fc640 1 -- 172.21.15.5:0/4179135585 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968031ba0 con 0x7f09780b2550 2023-11-24T06:16:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.169+0000 7f0977fff640 1 -- 172.21.15.5:0/4179135585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09780b2550 msgr2=0x7f0978070550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.169+0000 7f0977fff640 1 --2- 172.21.15.5:0/4179135585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09780b2550 0x7f0978070550 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f09680097a0 tx=0x7f096802b8b0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.169+0000 7f0977fff640 1 -- 172.21.15.5:0/4179135585 shutdown_connections 2023-11-24T06:16:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.169+0000 7f0977fff640 1 --2- 172.21.15.5:0/4179135585 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0978161700 0x7f09780ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.169+0000 7f0977fff640 1 --2- 172.21.15.5:0/4179135585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09780b2550 0x7f0978070550 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.169+0000 7f0977fff640 1 -- 172.21.15.5:0/4179135585 >> 172.21.15.5:0/4179135585 conn(0x7f09780b0840 msgr2=0x7f09780b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.169+0000 7f0977fff640 1 -- 172.21.15.5:0/4179135585 shutdown_connections 2023-11-24T06:16:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.169+0000 7f0977fff640 1 -- 172.21.15.5:0/4179135585 wait complete. 2023-11-24T06:16:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.171+0000 7f0977fff640 1 Processor -- start 2023-11-24T06:16:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.171+0000 7f0977fff640 1 -- start start 2023-11-24T06:16:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.172+0000 7f0977fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09780b2550 0x7f097815a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.172+0000 7f0977fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09780b2930 0x7f09781580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.172+0000 7f0977fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0978161700 0x7f09780aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.172+0000 7f0977fff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0978159490 con 0x7f0978161700 2023-11-24T06:16:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.172+0000 7f0977fff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0978158600 con 0x7f09780b2930 2023-11-24T06:16:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.172+0000 7f0977fff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0978158810 con 0x7f09780b2550 2023-11-24T06:16:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.173+0000 7f09777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0978161700 0x7f09780aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.173+0000 7f09777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0978161700 0x7f09780aeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42856/0 (socket says 172.21.15.5:42856) 2023-11-24T06:16:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.173+0000 7f09777fe640 1 -- 172.21.15.5:0/2433936083 learned_addr learned my addr 172.21.15.5:0/2433936083 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.173+0000 7f0976ffd640 1 --2- 172.21.15.5:0/2433936083 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09780b2550 0x7f097815a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.173+0000 7f096ffff640 1 --2- 172.21.15.5:0/2433936083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09780b2930 0x7f09781580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.173+0000 7f09777fe640 1 -- 172.21.15.5:0/2433936083 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09780b2550 msgr2=0x7f097815a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.173+0000 7f09777fe640 1 --2- 172.21.15.5:0/2433936083 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09780b2550 0x7f097815a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.173+0000 7f09777fe640 1 -- 172.21.15.5:0/2433936083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09780b2930 msgr2=0x7f09781580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.173+0000 7f09777fe640 1 --2- 172.21.15.5:0/2433936083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09780b2930 0x7f09781580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.173+0000 7f09777fe640 1 -- 172.21.15.5:0/2433936083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0968009400 con 0x7f0978161700 2023-11-24T06:16:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.174+0000 7f09777fe640 1 --2- 172.21.15.5:0/2433936083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0978161700 0x7f09780aeff0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f097000cc00 tx=0x7f097000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.174+0000 7f0974ff9640 1 -- 172.21.15.5:0/2433936083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097000ce10 con 0x7f0978161700 2023-11-24T06:16:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.174+0000 7f0974ff9640 1 -- 172.21.15.5:0/2433936083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0970003990 con 0x7f0978161700 2023-11-24T06:16:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.174+0000 7f0974ff9640 1 -- 172.21.15.5:0/2433936083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0970003d20 con 0x7f0978161700 2023-11-24T06:16:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.174+0000 7f0977fff640 1 -- 172.21.15.5:0/2433936083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0978158af0 con 0x7f0978161700 2023-11-24T06:16:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.174+0000 7f0977fff640 1 -- 172.21.15.5:0/2433936083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09781590b0 con 0x7f0978161700 2023-11-24T06:16:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.175+0000 7f0977fff640 1 -- 172.21.15.5:0/2433936083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f097804e840 con 0x7f0978161700 2023-11-24T06:16:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.176+0000 7f0974ff9640 1 -- 172.21.15.5:0/2433936083 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0970003e80 con 0x7f0978161700 2023-11-24T06:16:54.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.177+0000 7f0974ff9640 1 --2- 172.21.15.5:0/2433936083 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f095806ae40 0x7f095806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:54.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.177+0000 7f0974ff9640 1 -- 172.21.15.5:0/2433936083 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0970089990 con 0x7f0978161700 2023-11-24T06:16:54.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.177+0000 7f0976ffd640 1 --2- 172.21.15.5:0/2433936083 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f095806ae40 0x7f095806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.178+0000 7f0976ffd640 1 --2- 172.21.15.5:0/2433936083 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f095806ae40 0x7f095806d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0968036000 tx=0x7f096803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.182+0000 7f0974ff9640 1 -- 172.21.15.5:0/2433936083 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0970054a40 con 0x7f0978161700 2023-11-24T06:16:54.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.389+0000 7f0977fff640 1 -- 172.21.15.5:0/2433936083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f097806db90 con 0x7f0978161700 2023-11-24T06:16:54.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.390+0000 7f0974ff9640 1 -- 172.21.15.5:0/2433936083 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f097001d070 con 0x7f0978161700 2023-11-24T06:16:54.393 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2023-11-24T06:16:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.392+0000 7f0977fff640 1 -- 172.21.15.5:0/2433936083 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f095806ae40 msgr2=0x7f095806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.392+0000 7f0977fff640 1 --2- 172.21.15.5:0/2433936083 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f095806ae40 0x7f095806d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0968036000 tx=0x7f096803c040 comp rx=0 tx=0).stop 2023-11-24T06:16:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.392+0000 7f0977fff640 1 -- 172.21.15.5:0/2433936083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0978161700 msgr2=0x7f09780aeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.392+0000 7f0977fff640 1 --2- 172.21.15.5:0/2433936083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0978161700 0x7f09780aeff0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f097000cc00 tx=0x7f097000a5c0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.393+0000 7f0977fff640 1 -- 172.21.15.5:0/2433936083 shutdown_connections 2023-11-24T06:16:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.393+0000 7f0977fff640 1 --2- 172.21.15.5:0/2433936083 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f095806ae40 0x7f095806d300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.393+0000 7f0977fff640 1 --2- 172.21.15.5:0/2433936083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0978161700 0x7f09780aeff0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.393+0000 7f0977fff640 1 --2- 172.21.15.5:0/2433936083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f09780b2930 0x7f09781580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.393+0000 7f0977fff640 1 --2- 172.21.15.5:0/2433936083 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f09780b2550 0x7f097815a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.393+0000 7f0977fff640 1 -- 172.21.15.5:0/2433936083 >> 172.21.15.5:0/2433936083 conn(0x7f09780b0840 msgr2=0x7f097806cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.393+0000 7f0977fff640 1 -- 172.21.15.5:0/2433936083 shutdown_connections 2023-11-24T06:16:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.393+0000 7f0977fff640 1 -- 172.21.15.5:0/2433936083 wait complete. 2023-11-24T06:16:54.415 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi005.front.sepia.ceph.com... 2023-11-24T06:16:54.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:54.416 DEBUG:teuthology.orchestra.run.smithi005:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-24T06:16:54.518 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:16:54.518 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T06:16:54.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:54.518 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:16:54.577 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:16:54.577 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:16:54.641 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:16:54.641 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:16:54.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:54.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.932+0000 7f6fca997640 1 Processor -- start 2023-11-24T06:16:54.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.932+0000 7f6fca997640 1 -- start start 2023-11-24T06:16:54.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.933+0000 7f6fca997640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6fc4146010 0x7f6fc41463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:54.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.933+0000 7f6fca997640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fc40d0fe0 0x7f6fc406ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:54.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.933+0000 7f6fca997640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc415d970 con 0x7f6fc406efd0 2023-11-24T06:16:54.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.933+0000 7f6fca997640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc406f460 con 0x7f6fc4146010 2023-11-24T06:16:54.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.933+0000 7f6fca997640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc406b890 con 0x7f6fc40d0fe0 2023-11-24T06:16:54.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.934+0000 7f6fc3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6fc4146010 0x7f6fc41463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.934+0000 7f6fc8f34640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f6fc406efd0 0x7f6fc4160f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.5:47938/0 (socket says 172.21.15.5:47938) 2023-11-24T06:16:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.934+0000 7f6fc8f34640 1 -- 172.21.15.5:0/2949155819 learned_addr learned my addr 172.21.15.5:0/2949155819 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.934+0000 7f6fc37fe640 1 --2- 172.21.15.5:0/2949155819 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fc40d0fe0 0x7f6fc406ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.935+0000 7f6fc3fff640 1 -- 172.21.15.5:0/2949155819 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fc40d0fe0 msgr2=0x7f6fc406ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.935+0000 7f6fc3fff640 1 --2- 172.21.15.5:0/2949155819 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fc40d0fe0 0x7f6fc406ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.935+0000 7f6fc3fff640 1 -- 172.21.15.5:0/2949155819 >> v1:172.21.15.6:6789/0 conn(0x7f6fc406efd0 legacy=0x7f6fc4160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.935+0000 7f6fc3fff640 1 -- 172.21.15.5:0/2949155819 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc40d1550 con 0x7f6fc4146010 2023-11-24T06:16:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.936+0000 7f6fc3fff640 1 --2- 172.21.15.5:0/2949155819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc4146010 0x7f6fc41463f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40097a0 tx=0x7f6fb402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b1d55e53975b98 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.937+0000 7f6fc2ffd640 1 -- 172.21.15.5:0/2949155819 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4031830 con 0x7f6fc4146010 2023-11-24T06:16:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.937+0000 7f6fc2ffd640 1 -- 172.21.15.5:0/2949155819 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb4031990 con 0x7f6fc4146010 2023-11-24T06:16:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.937+0000 7f6fc2ffd640 1 -- 172.21.15.5:0/2949155819 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4031d20 con 0x7f6fc4146010 2023-11-24T06:16:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.937+0000 7f6fca997640 1 -- 172.21.15.5:0/2949155819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc4146010 msgr2=0x7f6fc41463f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.937+0000 7f6fca997640 1 --2- 172.21.15.5:0/2949155819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc4146010 0x7f6fc41463f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40097a0 tx=0x7f6fb402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.937+0000 7f6fca997640 1 -- 172.21.15.5:0/2949155819 shutdown_connections 2023-11-24T06:16:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.937+0000 7f6fca997640 1 --2- 172.21.15.5:0/2949155819 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fc40d0fe0 0x7f6fc406ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.937+0000 7f6fca997640 1 --2- 172.21.15.5:0/2949155819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc4146010 0x7f6fc41463f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.937+0000 7f6fca997640 1 -- 172.21.15.5:0/2949155819 >> 172.21.15.5:0/2949155819 conn(0x7f6fc40663d0 msgr2=0x7f6fc40667b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.938+0000 7f6fca997640 1 -- 172.21.15.5:0/2949155819 shutdown_connections 2023-11-24T06:16:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.938+0000 7f6fca997640 1 -- 172.21.15.5:0/2949155819 wait complete. 2023-11-24T06:16:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.939+0000 7f6fca997640 1 Processor -- start 2023-11-24T06:16:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.940+0000 7f6fca997640 1 -- start start 2023-11-24T06:16:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.940+0000 7f6fca997640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fc406efd0 0x7f6fc414e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.941+0000 7f6fca997640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc40d0fe0 0x7f6fc40d0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.941+0000 7f6fca997640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fc4146010 0x7f6fc40ce570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.941+0000 7f6fca997640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc414c980 con 0x7f6fc40d0fe0 2023-11-24T06:16:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.941+0000 7f6fca997640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40d0c60 con 0x7f6fc406efd0 2023-11-24T06:16:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.941+0000 7f6fca997640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc414ef50 con 0x7f6fc4146010 2023-11-24T06:16:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.941+0000 7f6fc37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc40d0fe0 0x7f6fc40d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.941+0000 7f6fc37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc40d0fe0 0x7f6fc40d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42874/0 (socket says 172.21.15.5:42874) 2023-11-24T06:16:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.941+0000 7f6fc37fe640 1 -- 172.21.15.5:0/2837081213 learned_addr learned my addr 172.21.15.5:0/2837081213 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.941+0000 7f6fc3fff640 1 --2- 172.21.15.5:0/2837081213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fc406efd0 0x7f6fc414e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.941+0000 7f6fc37fe640 1 -- 172.21.15.5:0/2837081213 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fc4146010 msgr2=0x7f6fc40ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.941+0000 7f6fc37fe640 1 --2- 172.21.15.5:0/2837081213 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fc4146010 0x7f6fc40ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.941+0000 7f6fc37fe640 1 -- 172.21.15.5:0/2837081213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fc406efd0 msgr2=0x7f6fc414e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.941+0000 7f6fc37fe640 1 --2- 172.21.15.5:0/2837081213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fc406efd0 0x7f6fc414e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.941+0000 7f6fc37fe640 1 -- 172.21.15.5:0/2837081213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb4009400 con 0x7f6fc40d0fe0 2023-11-24T06:16:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.942+0000 7f6fc37fe640 1 --2- 172.21.15.5:0/2837081213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc40d0fe0 0x7f6fc40d0720 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800e9b0 tx=0x7f6fb800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.942+0000 7f6fc0ff9640 1 -- 172.21.15.5:0/2837081213 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8009670 con 0x7f6fc40d0fe0 2023-11-24T06:16:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.942+0000 7f6fc0ff9640 1 -- 172.21.15.5:0/2837081213 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb800c9a0 con 0x7f6fc40d0fe0 2023-11-24T06:16:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.942+0000 7f6fc0ff9640 1 -- 172.21.15.5:0/2837081213 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb800cd50 con 0x7f6fc40d0fe0 2023-11-24T06:16:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.942+0000 7f6fca997640 1 -- 172.21.15.5:0/2837081213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc414f230 con 0x7f6fc40d0fe0 2023-11-24T06:16:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.942+0000 7f6fca997640 1 -- 172.21.15.5:0/2837081213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc414f7f0 con 0x7f6fc40d0fe0 2023-11-24T06:16:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.944+0000 7f6fca997640 1 -- 172.21.15.5:0/2837081213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc4074770 con 0x7f6fc40d0fe0 2023-11-24T06:16:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.944+0000 7f6fc0ff9640 1 -- 172.21.15.5:0/2837081213 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6fb8013470 con 0x7f6fc40d0fe0 2023-11-24T06:16:54.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.946+0000 7f6fc0ff9640 1 --2- 172.21.15.5:0/2837081213 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6fb006ae40 0x7f6fb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:54.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.946+0000 7f6fc0ff9640 1 -- 172.21.15.5:0/2837081213 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6fb80899f0 con 0x7f6fc40d0fe0 2023-11-24T06:16:54.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.947+0000 7f6fc3fff640 1 --2- 172.21.15.5:0/2837081213 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6fb006ae40 0x7f6fb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:54.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.947+0000 7f6fc3fff640 1 --2- 172.21.15.5:0/2837081213 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6fb006ae40 0x7f6fb006d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4036000 tx=0x7f6fb403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:54.951+0000 7f6fc0ff9640 1 -- 172.21.15.5:0/2837081213 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6fb8054970 con 0x7f6fc40d0fe0 2023-11-24T06:16:55.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.150+0000 7f6fca997640 1 -- 172.21.15.5:0/2837081213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fc414cac0 con 0x7f6fc40d0fe0 2023-11-24T06:16:55.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.153+0000 7f6fc0ff9640 1 -- 172.21.15.5:0/2837081213 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16658 (secure 0 0 0) 0x7f6fc414cac0 con 0x7f6fc40d0fe0 2023-11-24T06:16:55.157 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:55.157 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:16:33.958286+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:55.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.156+0000 7f6fca997640 1 -- 172.21.15.5:0/2837081213 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6fb006ae40 msgr2=0x7f6fb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.156+0000 7f6fca997640 1 --2- 172.21.15.5:0/2837081213 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6fb006ae40 0x7f6fb006d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4036000 tx=0x7f6fb403c040 comp rx=0 tx=0).stop 2023-11-24T06:16:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.156+0000 7f6fca997640 1 -- 172.21.15.5:0/2837081213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc40d0fe0 msgr2=0x7f6fc40d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.156+0000 7f6fca997640 1 --2- 172.21.15.5:0/2837081213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc40d0fe0 0x7f6fc40d0720 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800e9b0 tx=0x7f6fb800ee70 comp rx=0 tx=0).stop 2023-11-24T06:16:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.156+0000 7f6fca997640 1 -- 172.21.15.5:0/2837081213 shutdown_connections 2023-11-24T06:16:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.156+0000 7f6fca997640 1 --2- 172.21.15.5:0/2837081213 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f6fb006ae40 0x7f6fb006d300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.156+0000 7f6fca997640 1 --2- 172.21.15.5:0/2837081213 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6fc4146010 0x7f6fc40ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.156+0000 7f6fca997640 1 --2- 172.21.15.5:0/2837081213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6fc40d0fe0 0x7f6fc40d0720 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.156+0000 7f6fca997640 1 --2- 172.21.15.5:0/2837081213 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f6fc406efd0 0x7f6fc414e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.156+0000 7f6fca997640 1 -- 172.21.15.5:0/2837081213 >> 172.21.15.5:0/2837081213 conn(0x7f6fc40663d0 msgr2=0x7f6fc41410d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.157+0000 7f6fca997640 1 -- 172.21.15.5:0/2837081213 shutdown_connections 2023-11-24T06:16:55.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.157+0000 7f6fca997640 1 -- 172.21.15.5:0/2837081213 wait complete. 2023-11-24T06:16:55.178 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:55.179 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.407+0000 7f99c8d89640 1 Processor -- start 2023-11-24T06:16:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.408+0000 7f99c8d89640 1 -- start start 2023-11-24T06:16:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.409+0000 7f99c8d89640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99c40b48d0 0x7f99c40b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.409+0000 7f99c8d89640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c40b23e0 con 0x7f99c40b3210 2023-11-24T06:16:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.409+0000 7f99c8d89640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c40d41b0 con 0x7f99c4160bd0 2023-11-24T06:16:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.409+0000 7f99c8d89640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c40cdc70 con 0x7f99c40b48d0 2023-11-24T06:16:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.410+0000 7f99c2d9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f99c40b3210 0x7f99c4167290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46250/0 (socket says 172.21.15.5:46250) 2023-11-24T06:16:55.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.410+0000 7f99c2d9d640 1 -- 172.21.15.5:0/92298750 learned_addr learned my addr 172.21.15.5:0/92298750 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.410+0000 7f99c1d9b640 1 --2- 172.21.15.5:0/92298750 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99c40b48d0 0x7f99c40b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.410+0000 7f99c159a640 1 -- 172.21.15.5:0/92298750 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477839911 0 0) 0x7f99c40b23e0 con 0x7f99c40b3210 2023-11-24T06:16:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.410+0000 7f99c159a640 1 -- 172.21.15.5:0/92298750 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99ac003520 con 0x7f99c40b3210 2023-11-24T06:16:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.411+0000 7f99c159a640 1 -- 172.21.15.5:0/92298750 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 782255838 0 0) 0x7f99ac003520 con 0x7f99c40b3210 2023-11-24T06:16:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.411+0000 7f99c159a640 1 -- 172.21.15.5:0/92298750 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99c40b48d0 msgr2=0x7f99c40b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.411+0000 7f99c159a640 1 --2- 172.21.15.5:0/92298750 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99c40b48d0 0x7f99c40b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.411+0000 7f99c159a640 1 -- 172.21.15.5:0/92298750 >> v1:172.21.15.6:6789/0 conn(0x7f99c4160bd0 legacy=0x7f99c40b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.411+0000 7f99c159a640 1 -- 172.21.15.5:0/92298750 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c4160fb0 con 0x7f99c40b3210 2023-11-24T06:16:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.411+0000 7f99c159a640 1 -- 172.21.15.5:0/92298750 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f99bc0019f0 con 0x7f99c40b3210 2023-11-24T06:16:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.411+0000 7f99c159a640 1 -- 172.21.15.5:0/92298750 >> v1:172.21.15.5:6789/0 conn(0x7f99c40b3210 legacy=0x7f99c4167290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.411+0000 7f99c159a640 1 --2- 172.21.15.5:0/92298750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99ac0039a0 0x7f99ac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:55.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.411+0000 7f99c159a640 1 --2- 172.21.15.5:0/92298750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99ac0047c0 0x7f99ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.411+0000 7f99c159a640 1 --2- 172.21.15.5:0/92298750 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99c40b48d0 0x7f99ac0051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.411+0000 7f99c159a640 1 -- 172.21.15.5:0/92298750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c40cdc70 con 0x7f99ac0039a0 2023-11-24T06:16:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.411+0000 7f99c159a640 1 -- 172.21.15.5:0/92298750 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac00ba90 con 0x7f99ac0047c0 2023-11-24T06:16:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.411+0000 7f99c159a640 1 -- 172.21.15.5:0/92298750 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac00bc50 con 0x7f99c40b48d0 2023-11-24T06:16:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.411+0000 7f99c259c640 1 --2- 172.21.15.5:0/92298750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99ac0039a0 0x7f99ac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.412+0000 7f99c2d9d640 1 --2- 172.21.15.5:0/92298750 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99c40b48d0 0x7f99ac0051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.412+0000 7f99c1d9b640 1 --2- 172.21.15.5:0/92298750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99ac0047c0 0x7f99ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.412+0000 7f99c259c640 1 -- 172.21.15.5:0/92298750 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99c40b48d0 msgr2=0x7f99ac0051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.412+0000 7f99c259c640 1 --2- 172.21.15.5:0/92298750 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99c40b48d0 0x7f99ac0051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.412+0000 7f99c259c640 1 -- 172.21.15.5:0/92298750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99ac0047c0 msgr2=0x7f99ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.412+0000 7f99c259c640 1 --2- 172.21.15.5:0/92298750 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99ac0047c0 0x7f99ac004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.412+0000 7f99c259c640 1 -- 172.21.15.5:0/92298750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac003520 con 0x7f99ac0039a0 2023-11-24T06:16:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.413+0000 7f99c259c640 1 --2- 172.21.15.5:0/92298750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99ac0039a0 0x7f99ac003da0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f99b4001860 tx=0x7f99ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fad3d380e9cf27 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.413+0000 7f99c159a640 1 -- 172.21.15.5:0/92298750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4033630 con 0x7f99ac0039a0 2023-11-24T06:16:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.413+0000 7f99c159a640 1 -- 172.21.15.5:0/92298750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b40337b0 con 0x7f99ac0039a0 2023-11-24T06:16:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.413+0000 7f99c159a640 1 -- 172.21.15.5:0/92298750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4033b60 con 0x7f99ac0039a0 2023-11-24T06:16:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.414+0000 7f99c8d89640 1 -- 172.21.15.5:0/92298750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99ac0039a0 msgr2=0x7f99ac003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.414+0000 7f99c8d89640 1 --2- 172.21.15.5:0/92298750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99ac0039a0 0x7f99ac003da0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f99b4001860 tx=0x7f99ac0057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.414+0000 7f99c2d9d640 1 -- 172.21.15.5:0/92298750 reap_dead start 2023-11-24T06:16:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.414+0000 7f99c8d89640 1 -- 172.21.15.5:0/92298750 shutdown_connections 2023-11-24T06:16:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.414+0000 7f99c8d89640 1 -- 172.21.15.5:0/92298750 >> 172.21.15.5:0/92298750 conn(0x7f99c409c5a0 msgr2=0x7f99c40d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.414+0000 7f99c8d89640 1 -- 172.21.15.5:0/92298750 shutdown_connections 2023-11-24T06:16:55.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.414+0000 7f99c8d89640 1 -- 172.21.15.5:0/92298750 wait complete. 2023-11-24T06:16:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.416+0000 7f99c8d89640 1 Processor -- start 2023-11-24T06:16:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.416+0000 7f99c8d89640 1 -- start start 2023-11-24T06:16:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.417+0000 7f99c8d89640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99ac0039a0 0x7f99c4154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.417+0000 7f99c8d89640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99c40b7070 0x7f99c40b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.417+0000 7f99c8d89640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99c415ec00 0x7f99c415f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.417+0000 7f99c8d89640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c415f5f0 con 0x7f99ac0039a0 2023-11-24T06:16:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.417+0000 7f99c8d89640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c40b8740 con 0x7f99c415ec00 2023-11-24T06:16:55.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.417+0000 7f99c8d89640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c40b7b30 con 0x7f99c40b7070 2023-11-24T06:16:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.417+0000 7f99c259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99ac0039a0 0x7f99c4154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.417+0000 7f99c259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99ac0039a0 0x7f99c4154a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42880/0 (socket says 172.21.15.5:42880) 2023-11-24T06:16:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.417+0000 7f99c259c640 1 -- 172.21.15.5:0/2096681717 learned_addr learned my addr 172.21.15.5:0/2096681717 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.418+0000 7f99c259c640 1 -- 172.21.15.5:0/2096681717 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99c40b7070 msgr2=0x7f99c40b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.418+0000 7f99c259c640 1 --2- 172.21.15.5:0/2096681717 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99c40b7070 0x7f99c40b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.418+0000 7f99c259c640 1 -- 172.21.15.5:0/2096681717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99c415ec00 msgr2=0x7f99c415f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.418+0000 7f99c259c640 1 --2- 172.21.15.5:0/2096681717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99c415ec00 0x7f99c415f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.418+0000 7f99c259c640 1 -- 172.21.15.5:0/2096681717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c40b7e10 con 0x7f99ac0039a0 2023-11-24T06:16:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.418+0000 7f99c259c640 1 --2- 172.21.15.5:0/2096681717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99ac0039a0 0x7f99c4154a10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f99b4008fd0 tx=0x7f99b4001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.418+0000 7f99b2ffd640 1 -- 172.21.15.5:0/2096681717 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4007050 con 0x7f99ac0039a0 2023-11-24T06:16:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.418+0000 7f99b2ffd640 1 -- 172.21.15.5:0/2096681717 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b40071b0 con 0x7f99ac0039a0 2023-11-24T06:16:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.418+0000 7f99b2ffd640 1 -- 172.21.15.5:0/2096681717 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4007540 con 0x7f99ac0039a0 2023-11-24T06:16:55.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.419+0000 7f99c8d89640 1 -- 172.21.15.5:0/2096681717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c40b80f0 con 0x7f99ac0039a0 2023-11-24T06:16:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.419+0000 7f99c8d89640 1 -- 172.21.15.5:0/2096681717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c40cc730 con 0x7f99ac0039a0 2023-11-24T06:16:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.420+0000 7f99c8d89640 1 -- 172.21.15.5:0/2096681717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c40a1210 con 0x7f99ac0039a0 2023-11-24T06:16:55.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.421+0000 7f99b2ffd640 1 -- 172.21.15.5:0/2096681717 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f99b40067d0 con 0x7f99ac0039a0 2023-11-24T06:16:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.422+0000 7f99b2ffd640 1 --2- 172.21.15.5:0/2096681717 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f99a406ae70 0x7f99a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.422+0000 7f99b2ffd640 1 -- 172.21.15.5:0/2096681717 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f99b40ab620 con 0x7f99ac0039a0 2023-11-24T06:16:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.422+0000 7f99c1d9b640 1 --2- 172.21.15.5:0/2096681717 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f99a406ae70 0x7f99a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.423+0000 7f99c1d9b640 1 --2- 172.21.15.5:0/2096681717 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f99a406ae70 0x7f99a406d330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f99c40c6840 tx=0x7f99b8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:55.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.426+0000 7f99b2ffd640 1 -- 172.21.15.5:0/2096681717 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f99b40767f0 con 0x7f99ac0039a0 2023-11-24T06:16:55.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.619+0000 7f99c8d89640 1 -- 172.21.15.5:0/2096681717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99c40cd870 con 0x7f99ac0039a0 2023-11-24T06:16:55.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.620+0000 7f99b2ffd640 1 -- 172.21.15.5:0/2096681717 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f99b4079e10 con 0x7f99ac0039a0 2023-11-24T06:16:55.623 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:55.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.623+0000 7f99c8d89640 1 -- 172.21.15.5:0/2096681717 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f99a406ae70 msgr2=0x7f99a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.623+0000 7f99c8d89640 1 --2- 172.21.15.5:0/2096681717 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f99a406ae70 0x7f99a406d330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f99c40c6840 tx=0x7f99b8008040 comp rx=0 tx=0).stop 2023-11-24T06:16:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.623+0000 7f99c8d89640 1 -- 172.21.15.5:0/2096681717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99ac0039a0 msgr2=0x7f99c4154a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.623+0000 7f99c8d89640 1 --2- 172.21.15.5:0/2096681717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99ac0039a0 0x7f99c4154a10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f99b4008fd0 tx=0x7f99b4001230 comp rx=0 tx=0).stop 2023-11-24T06:16:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.623+0000 7f99c8d89640 1 -- 172.21.15.5:0/2096681717 shutdown_connections 2023-11-24T06:16:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.623+0000 7f99c8d89640 1 --2- 172.21.15.5:0/2096681717 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f99a406ae70 0x7f99a406d330 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.623+0000 7f99c8d89640 1 --2- 172.21.15.5:0/2096681717 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f99c40b7070 0x7f99c40b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.623+0000 7f99c8d89640 1 --2- 172.21.15.5:0/2096681717 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f99c415ec00 0x7f99c415f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.623+0000 7f99c8d89640 1 --2- 172.21.15.5:0/2096681717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99ac0039a0 0x7f99c4154a10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.623+0000 7f99c8d89640 1 -- 172.21.15.5:0/2096681717 >> 172.21.15.5:0/2096681717 conn(0x7f99c409c5a0 msgr2=0x7f99c40d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:55.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.624+0000 7f99c8d89640 1 -- 172.21.15.5:0/2096681717 shutdown_connections 2023-11-24T06:16:55.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.624+0000 7f99c8d89640 1 -- 172.21.15.5:0/2096681717 wait complete. 2023-11-24T06:16:55.646 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:55.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.871+0000 7fb1c342e640 1 Processor -- start 2023-11-24T06:16:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.871+0000 7fb1c342e640 1 -- start start 2023-11-24T06:16:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.873+0000 7fb1c342e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb1bc15f270 0x7fb1bc0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.873+0000 7fb1c342e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb1bc06d9e0 0x7fb1bc0cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.873+0000 7fb1c342e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1bc15f930 con 0x7fb1bc144770 2023-11-24T06:16:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.873+0000 7fb1c342e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1bc0b43b0 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.873+0000 7fb1c342e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1bc0b3b10 con 0x7fb1bc15f270 2023-11-24T06:16:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.873+0000 7fb1c2c2d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb1bc06d9e0 0x7fb1bc0cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.874+0000 7fb1c2c2d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb1bc06d9e0 0x7fb1bc0cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42882/0 (socket says 172.21.15.5:42882) 2023-11-24T06:16:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.874+0000 7fb1c2c2d640 1 -- 172.21.15.5:0/4284933330 learned_addr learned my addr 172.21.15.5:0/4284933330 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.874+0000 7fb1c1c2b640 1 --2- 172.21.15.5:0/4284933330 >> v2:172.21.15.6:3300/0 conn(0x7fb1bc15f270 0x7fb1bc0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.874+0000 7fb1c2c2d640 1 -- 172.21.15.5:0/4284933330 >> v2:172.21.15.6:3300/0 conn(0x7fb1bc15f270 msgr2=0x7fb1bc0cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.874+0000 7fb1c2c2d640 1 --2- 172.21.15.5:0/4284933330 >> v2:172.21.15.6:3300/0 conn(0x7fb1bc15f270 0x7fb1bc0cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.874+0000 7fb1c2c2d640 1 -- 172.21.15.5:0/4284933330 >> v1:172.21.15.6:6789/0 conn(0x7fb1bc144770 legacy=0x7fb1bc15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.874+0000 7fb1c2c2d640 1 -- 172.21.15.5:0/4284933330 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc0b3df0 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.875+0000 7fb1c2c2d640 1 --2- 172.21.15.5:0/4284933330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc06d9e0 0x7fb1bc0cd9e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0ae900 tx=0x7fb1b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9722618547d9e7a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.875+0000 7fb1c142a640 1 -- 172.21.15.5:0/4284933330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b402ca80 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.876+0000 7fb1c142a640 1 -- 172.21.15.5:0/4284933330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b402cbe0 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.876+0000 7fb1c142a640 1 -- 172.21.15.5:0/4284933330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40343f0 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.876+0000 7fb1c342e640 1 -- 172.21.15.5:0/4284933330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc06d9e0 msgr2=0x7fb1bc0cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.876+0000 7fb1c342e640 1 --2- 172.21.15.5:0/4284933330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc06d9e0 0x7fb1bc0cd9e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0ae900 tx=0x7fb1b402bee0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.876+0000 7fb1c342e640 1 -- 172.21.15.5:0/4284933330 shutdown_connections 2023-11-24T06:16:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.876+0000 7fb1c342e640 1 --2- 172.21.15.5:0/4284933330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc06d9e0 0x7fb1bc0cd9e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.876+0000 7fb1c342e640 1 --2- 172.21.15.5:0/4284933330 >> v2:172.21.15.6:3300/0 conn(0x7fb1bc15f270 0x7fb1bc0cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.876+0000 7fb1c342e640 1 -- 172.21.15.5:0/4284933330 >> 172.21.15.5:0/4284933330 conn(0x7fb1bc0cec10 msgr2=0x7fb1bc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:55.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.876+0000 7fb1c342e640 1 -- 172.21.15.5:0/4284933330 shutdown_connections 2023-11-24T06:16:55.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.876+0000 7fb1c342e640 1 -- 172.21.15.5:0/4284933330 wait complete. 2023-11-24T06:16:55.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.878+0000 7fb1c342e640 1 Processor -- start 2023-11-24T06:16:55.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.879+0000 7fb1c342e640 1 -- start start 2023-11-24T06:16:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.880+0000 7fb1c342e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc06d9e0 0x7fb1bc159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.880+0000 7fb1c342e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1bc144770 0x7fb1bc15b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.880+0000 7fb1c342e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1bc15f270 0x7fb1bc13cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.880+0000 7fb1c342e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc15cd30 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.880+0000 7fb1c342e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc15c0b0 con 0x7fb1bc15f270 2023-11-24T06:16:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.880+0000 7fb1c342e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc15c390 con 0x7fb1bc144770 2023-11-24T06:16:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.880+0000 7fb1c242c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc06d9e0 0x7fb1bc159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.880+0000 7fb1c242c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc06d9e0 0x7fb1bc159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42898/0 (socket says 172.21.15.5:42898) 2023-11-24T06:16:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.880+0000 7fb1c242c640 1 -- 172.21.15.5:0/2859638128 learned_addr learned my addr 172.21.15.5:0/2859638128 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.880+0000 7fb1c1c2b640 1 --2- 172.21.15.5:0/2859638128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1bc144770 0x7fb1bc15b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.881+0000 7fb1c242c640 1 -- 172.21.15.5:0/2859638128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1bc144770 msgr2=0x7fb1bc15b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.881+0000 7fb1c242c640 1 --2- 172.21.15.5:0/2859638128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1bc144770 0x7fb1bc15b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.881+0000 7fb1c2c2d640 1 --2- 172.21.15.5:0/2859638128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1bc15f270 0x7fb1bc13cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.881+0000 7fb1c242c640 1 -- 172.21.15.5:0/2859638128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1bc15f270 msgr2=0x7fb1bc13cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.881+0000 7fb1c242c640 1 --2- 172.21.15.5:0/2859638128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1bc15f270 0x7fb1bc13cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.881+0000 7fb1c242c640 1 -- 172.21.15.5:0/2859638128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b4009b30 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.881+0000 7fb1c242c640 1 --2- 172.21.15.5:0/2859638128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc06d9e0 0x7fb1bc159c90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800c410 tx=0x7fb1b800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:55.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.881+0000 7fb1b2ffd640 1 -- 172.21.15.5:0/2859638128 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80035f0 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.881+0000 7fb1b2ffd640 1 -- 172.21.15.5:0/2859638128 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b8003750 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.881+0000 7fb1b2ffd640 1 -- 172.21.15.5:0/2859638128 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8003b40 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.881+0000 7fb1c342e640 1 -- 172.21.15.5:0/2859638128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1bc15c670 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.882+0000 7fb1c342e640 1 -- 172.21.15.5:0/2859638128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc13c430 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.883+0000 7fb1c342e640 1 -- 172.21.15.5:0/2859638128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1bc13c5c0 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.883+0000 7fb1b2ffd640 1 -- 172.21.15.5:0/2859638128 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb1b8003ca0 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.884+0000 7fb1b2ffd640 1 --2- 172.21.15.5:0/2859638128 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb1a006ae40 0x7fb1a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.884+0000 7fb1b2ffd640 1 -- 172.21.15.5:0/2859638128 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb1b80888c0 con 0x7fb1bc06d9e0 2023-11-24T06:16:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.885+0000 7fb1c1c2b640 1 --2- 172.21.15.5:0/2859638128 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb1a006ae40 0x7fb1a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:55.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.885+0000 7fb1c1c2b640 1 --2- 172.21.15.5:0/2859638128 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb1a006ae40 0x7fb1a006d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac006fd0 tx=0x7fb1ac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:55.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:55.889+0000 7fb1b2ffd640 1 -- 172.21.15.5:0/2859638128 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb1b80538d0 con 0x7fb1bc06d9e0 2023-11-24T06:16:56.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.076+0000 7fb1c342e640 1 -- 172.21.15.5:0/2859638128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1bc04e840 con 0x7fb1bc06d9e0 2023-11-24T06:16:56.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.077+0000 7fb1b2ffd640 1 -- 172.21.15.5:0/2859638128 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb1b8056ef0 con 0x7fb1bc06d9e0 2023-11-24T06:16:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.079+0000 7fb1c342e640 1 -- 172.21.15.5:0/2859638128 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb1a006ae40 msgr2=0x7fb1a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.079+0000 7fb1c342e640 1 --2- 172.21.15.5:0/2859638128 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb1a006ae40 0x7fb1a006d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac006fd0 tx=0x7fb1ac008040 comp rx=0 tx=0).stop 2023-11-24T06:16:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.079+0000 7fb1c342e640 1 -- 172.21.15.5:0/2859638128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc06d9e0 msgr2=0x7fb1bc159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.079+0000 7fb1c342e640 1 --2- 172.21.15.5:0/2859638128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc06d9e0 0x7fb1bc159c90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb1b800c410 tx=0x7fb1b800c8d0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.079+0000 7fb1c342e640 1 -- 172.21.15.5:0/2859638128 shutdown_connections 2023-11-24T06:16:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.079+0000 7fb1c342e640 1 --2- 172.21.15.5:0/2859638128 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb1a006ae40 0x7fb1a006d300 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.079+0000 7fb1c342e640 1 --2- 172.21.15.5:0/2859638128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb1bc144770 0x7fb1bc15b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.079+0000 7fb1c342e640 1 --2- 172.21.15.5:0/2859638128 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb1bc15f270 0x7fb1bc13cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.079+0000 7fb1c342e640 1 --2- 172.21.15.5:0/2859638128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1bc06d9e0 0x7fb1bc159c90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.079+0000 7fb1c342e640 1 -- 172.21.15.5:0/2859638128 >> 172.21.15.5:0/2859638128 conn(0x7fb1bc0cec10 msgr2=0x7fb1bc0ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.079+0000 7fb1c342e640 1 -- 172.21.15.5:0/2859638128 shutdown_connections 2023-11-24T06:16:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.079+0000 7fb1c342e640 1 -- 172.21.15.5:0/2859638128 wait complete. 2023-11-24T06:16:56.101 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:56.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.326+0000 7fa1e0947640 1 Processor -- start 2023-11-24T06:16:56.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.327+0000 7fa1e0947640 1 -- start start 2023-11-24T06:16:56.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.328+0000 7fa1e0947640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1dc0d1d90 0x7fa1dc0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:56.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.328+0000 7fa1e0947640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc0c6bf0 con 0x7fa1dc15d3b0 2023-11-24T06:16:56.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.328+0000 7fa1e0947640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc0d13d0 con 0x7fa1dc0d0fa0 2023-11-24T06:16:56.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.328+0000 7fa1e0947640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0cedc0 con 0x7fa1dc0d1d90 2023-11-24T06:16:56.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.329+0000 7fa1db7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa1dc15d3b0 0x7fa1dc0d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46258/0 (socket says 172.21.15.5:46258) 2023-11-24T06:16:56.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.329+0000 7fa1db7fe640 1 -- 172.21.15.5:0/2026747617 learned_addr learned my addr 172.21.15.5:0/2026747617 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.329+0000 7fa1d3fff640 1 --2- 172.21.15.5:0/2026747617 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1dc0d1d90 0x7fa1dc0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.329+0000 7fa1daffd640 1 -- 172.21.15.5:0/2026747617 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720728073 0 0) 0x7fa1dc0c6bf0 con 0x7fa1dc15d3b0 2023-11-24T06:16:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.329+0000 7fa1daffd640 1 -- 172.21.15.5:0/2026747617 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1c8003520 con 0x7fa1dc15d3b0 2023-11-24T06:16:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.329+0000 7fa1daffd640 1 -- 172.21.15.5:0/2026747617 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3529253854 0 0) 0x7fa1c8003520 con 0x7fa1dc15d3b0 2023-11-24T06:16:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.329+0000 7fa1daffd640 1 -- 172.21.15.5:0/2026747617 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1dc0d1d90 msgr2=0x7fa1dc0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.329+0000 7fa1daffd640 1 --2- 172.21.15.5:0/2026747617 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1dc0d1d90 0x7fa1dc0ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.329+0000 7fa1daffd640 1 -- 172.21.15.5:0/2026747617 >> v1:172.21.15.6:6789/0 conn(0x7fa1dc0d0fa0 legacy=0x7fa1dc0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.329+0000 7fa1daffd640 1 -- 172.21.15.5:0/2026747617 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc0c6e40 con 0x7fa1dc15d3b0 2023-11-24T06:16:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.330+0000 7fa1daffd640 1 -- 172.21.15.5:0/2026747617 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fa1cc001670 con 0x7fa1dc15d3b0 2023-11-24T06:16:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.330+0000 7fa1daffd640 1 -- 172.21.15.5:0/2026747617 >> v1:172.21.15.5:6789/0 conn(0x7fa1dc15d3b0 legacy=0x7fa1dc0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.330+0000 7fa1daffd640 1 --2- 172.21.15.5:0/2026747617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1c8003a10 0x7fa1c8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.330+0000 7fa1daffd640 1 --2- 172.21.15.5:0/2026747617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c80047c0 0x7fa1c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.330+0000 7fa1daffd640 1 --2- 172.21.15.5:0/2026747617 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1dc0d1d90 0x7fa1c8005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.330+0000 7fa1daffd640 1 -- 172.21.15.5:0/2026747617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0cedc0 con 0x7fa1c8003a10 2023-11-24T06:16:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.330+0000 7fa1daffd640 1 -- 172.21.15.5:0/2026747617 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c800bb70 con 0x7fa1c80047c0 2023-11-24T06:16:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.330+0000 7fa1daffd640 1 -- 172.21.15.5:0/2026747617 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c800bd30 con 0x7fa1dc0d1d90 2023-11-24T06:16:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.330+0000 7fa1d3fff640 1 --2- 172.21.15.5:0/2026747617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c80047c0 0x7fa1c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.331+0000 7fa1dbfff640 1 --2- 172.21.15.5:0/2026747617 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1dc0d1d90 0x7fa1c8005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.331+0000 7fa1db7fe640 1 --2- 172.21.15.5:0/2026747617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1c8003a10 0x7fa1c8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.331+0000 7fa1d3fff640 1 -- 172.21.15.5:0/2026747617 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1dc0d1d90 msgr2=0x7fa1c8005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.331+0000 7fa1d3fff640 1 --2- 172.21.15.5:0/2026747617 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1dc0d1d90 0x7fa1c8005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.331+0000 7fa1d3fff640 1 -- 172.21.15.5:0/2026747617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1c8003a10 msgr2=0x7fa1c8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.331+0000 7fa1d3fff640 1 --2- 172.21.15.5:0/2026747617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1c8003a10 0x7fa1c8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.331+0000 7fa1d3fff640 1 -- 172.21.15.5:0/2026747617 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c8003520 con 0x7fa1c80047c0 2023-11-24T06:16:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.332+0000 7fa1d3fff640 1 --2- 172.21.15.5:0/2026747617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c80047c0 0x7fa1c8004c40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009a90 tx=0x7fa1c80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cadc3d00d6e485dd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.333+0000 7fa1daffd640 1 -- 172.21.15.5:0/2026747617 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c402c680 con 0x7fa1c80047c0 2023-11-24T06:16:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.333+0000 7fa1daffd640 1 -- 172.21.15.5:0/2026747617 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c402c7e0 con 0x7fa1c80047c0 2023-11-24T06:16:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.333+0000 7fa1daffd640 1 -- 172.21.15.5:0/2026747617 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c402cb70 con 0x7fa1c80047c0 2023-11-24T06:16:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.334+0000 7fa1e0947640 1 -- 172.21.15.5:0/2026747617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c80047c0 msgr2=0x7fa1c8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.334+0000 7fa1e0947640 1 --2- 172.21.15.5:0/2026747617 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c80047c0 0x7fa1c8004c40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009a90 tx=0x7fa1c80057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.334+0000 7fa1dbfff640 1 -- 172.21.15.5:0/2026747617 reap_dead start 2023-11-24T06:16:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.334+0000 7fa1e0947640 1 -- 172.21.15.5:0/2026747617 shutdown_connections 2023-11-24T06:16:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.334+0000 7fa1e0947640 1 -- 172.21.15.5:0/2026747617 >> 172.21.15.5:0/2026747617 conn(0x7fa1dc0d30a0 msgr2=0x7fa1dc0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.334+0000 7fa1e0947640 1 -- 172.21.15.5:0/2026747617 shutdown_connections 2023-11-24T06:16:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.334+0000 7fa1e0947640 1 -- 172.21.15.5:0/2026747617 wait complete. 2023-11-24T06:16:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.336+0000 7fa1e0947640 1 Processor -- start 2023-11-24T06:16:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.336+0000 7fa1e0947640 1 -- start start 2023-11-24T06:16:56.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.337+0000 7fa1e0947640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c80047c0 0x7fa1dc15c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.337+0000 7fa1e0947640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc09ea30 0x7fa1dc0cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.337+0000 7fa1e0947640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1dc0cc7e0 0x7fa1dc0b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.337+0000 7fa1e0947640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc09e610 con 0x7fa1dc09ea30 2023-11-24T06:16:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.337+0000 7fa1e0947640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc09d7a0 con 0x7fa1c80047c0 2023-11-24T06:16:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.337+0000 7fa1e0947640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0a0880 con 0x7fa1dc0cc7e0 2023-11-24T06:16:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.337+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc09ea30 0x7fa1dc0cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.337+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc09ea30 0x7fa1dc0cc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42912/0 (socket says 172.21.15.5:42912) 2023-11-24T06:16:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.337+0000 7fa1d3fff640 1 -- 172.21.15.5:0/3998751029 learned_addr learned my addr 172.21.15.5:0/3998751029 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.338+0000 7fa1dbfff640 1 --2- 172.21.15.5:0/3998751029 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1dc0cc7e0 0x7fa1dc0b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.338+0000 7fa1db7fe640 1 --2- 172.21.15.5:0/3998751029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c80047c0 0x7fa1dc15c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.338+0000 7fa1d3fff640 1 -- 172.21.15.5:0/3998751029 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1dc0cc7e0 msgr2=0x7fa1dc0b82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.338+0000 7fa1d3fff640 1 --2- 172.21.15.5:0/3998751029 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1dc0cc7e0 0x7fa1dc0b82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.338+0000 7fa1d3fff640 1 -- 172.21.15.5:0/3998751029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c80047c0 msgr2=0x7fa1dc15c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.338+0000 7fa1d3fff640 1 --2- 172.21.15.5:0/3998751029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c80047c0 0x7fa1dc15c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.338+0000 7fa1d3fff640 1 -- 172.21.15.5:0/3998751029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc09d990 con 0x7fa1dc09ea30 2023-11-24T06:16:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.338+0000 7fa1d3fff640 1 --2- 172.21.15.5:0/3998751029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc09ea30 0x7fa1dc0cc2a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000c00 tx=0x7fa1c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.339+0000 7fa1d8ff9640 1 -- 172.21.15.5:0/3998751029 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4003610 con 0x7fa1dc09ea30 2023-11-24T06:16:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.339+0000 7fa1d8ff9640 1 -- 172.21.15.5:0/3998751029 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c4003770 con 0x7fa1dc09ea30 2023-11-24T06:16:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.339+0000 7fa1d8ff9640 1 -- 172.21.15.5:0/3998751029 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4003b20 con 0x7fa1dc09ea30 2023-11-24T06:16:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.339+0000 7fa1e0947640 1 -- 172.21.15.5:0/3998751029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc09dc70 con 0x7fa1dc09ea30 2023-11-24T06:16:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.339+0000 7fa1e0947640 1 -- 172.21.15.5:0/3998751029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc09e230 con 0x7fa1dc09ea30 2023-11-24T06:16:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.340+0000 7fa1e0947640 1 -- 172.21.15.5:0/3998751029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc09e3c0 con 0x7fa1dc09ea30 2023-11-24T06:16:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.341+0000 7fa1d8ff9640 1 -- 172.21.15.5:0/3998751029 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1c403b070 con 0x7fa1dc09ea30 2023-11-24T06:16:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.342+0000 7fa1d8ff9640 1 --2- 172.21.15.5:0/3998751029 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa1bc06ae70 0x7fa1bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.342+0000 7fa1d8ff9640 1 -- 172.21.15.5:0/3998751029 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa1c40aaca0 con 0x7fa1dc09ea30 2023-11-24T06:16:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.342+0000 7fa1db7fe640 1 --2- 172.21.15.5:0/3998751029 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa1bc06ae70 0x7fa1bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.343+0000 7fa1db7fe640 1 --2- 172.21.15.5:0/3998751029 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa1bc06ae70 0x7fa1bc06d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc007c40 tx=0x7fa1cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.347+0000 7fa1d8ff9640 1 -- 172.21.15.5:0/3998751029 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1c4075cb0 con 0x7fa1dc09ea30 2023-11-24T06:16:56.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.541+0000 7fa1e0947640 1 -- 172.21.15.5:0/3998751029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa1dc04e840 con 0x7fa1dc09ea30 2023-11-24T06:16:56.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.542+0000 7fa1d8ff9640 1 -- 172.21.15.5:0/3998751029 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa1c40390a0 con 0x7fa1dc09ea30 2023-11-24T06:16:56.545 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:56.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.545+0000 7fa1e0947640 1 -- 172.21.15.5:0/3998751029 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa1bc06ae70 msgr2=0x7fa1bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:56.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.545+0000 7fa1e0947640 1 --2- 172.21.15.5:0/3998751029 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa1bc06ae70 0x7fa1bc06d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc007c40 tx=0x7fa1cc009040 comp rx=0 tx=0).stop 2023-11-24T06:16:56.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.545+0000 7fa1e0947640 1 -- 172.21.15.5:0/3998751029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc09ea30 msgr2=0x7fa1dc0cc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:56.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.545+0000 7fa1e0947640 1 --2- 172.21.15.5:0/3998751029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc09ea30 0x7fa1dc0cc2a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000c00 tx=0x7fa1c4004730 comp rx=0 tx=0).stop 2023-11-24T06:16:56.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.545+0000 7fa1e0947640 1 -- 172.21.15.5:0/3998751029 shutdown_connections 2023-11-24T06:16:56.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.545+0000 7fa1e0947640 1 --2- 172.21.15.5:0/3998751029 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa1bc06ae70 0x7fa1bc06d330 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.545+0000 7fa1e0947640 1 --2- 172.21.15.5:0/3998751029 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1dc0cc7e0 0x7fa1dc0b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.545+0000 7fa1e0947640 1 --2- 172.21.15.5:0/3998751029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1dc09ea30 0x7fa1dc0cc2a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.545+0000 7fa1e0947640 1 --2- 172.21.15.5:0/3998751029 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c80047c0 0x7fa1dc15c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.545+0000 7fa1e0947640 1 -- 172.21.15.5:0/3998751029 >> 172.21.15.5:0/3998751029 conn(0x7fa1dc0d30a0 msgr2=0x7fa1dc06af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:56.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.545+0000 7fa1e0947640 1 -- 172.21.15.5:0/3998751029 shutdown_connections 2023-11-24T06:16:56.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.545+0000 7fa1e0947640 1 -- 172.21.15.5:0/3998751029 wait complete. 2023-11-24T06:16:56.569 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:16:56.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.791+0000 7fb40ffff640 1 Processor -- start 2023-11-24T06:16:56.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.792+0000 7fb40ffff640 1 -- start start 2023-11-24T06:16:56.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.794+0000 7fb40ffff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb410146370 0x7fb41006c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:56.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.794+0000 7fb40ffff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb410144ed0 con 0x7fb4100707e0 2023-11-24T06:16:56.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.794+0000 7fb40ffff640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41006d040 con 0x7fb4101449d0 2023-11-24T06:16:56.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.794+0000 7fb40ffff640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41006d320 con 0x7fb410146370 2023-11-24T06:16:56.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.794+0000 7fb407fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb4100707e0 0x7fb41006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46270/0 (socket says 172.21.15.5:46270) 2023-11-24T06:16:56.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.794+0000 7fb407fff640 1 -- 172.21.15.5:0/2042825947 learned_addr learned my addr 172.21.15.5:0/2042825947 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:56.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.794+0000 7fb40f7fe640 1 --2- 172.21.15.5:0/2042825947 >> v2:172.21.15.6:3300/0 conn(0x7fb410146370 0x7fb41006c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:56.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.795+0000 7fb40e7fc640 1 -- 172.21.15.5:0/2042825947 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230288096 0 0) 0x7fb410144ed0 con 0x7fb4100707e0 2023-11-24T06:16:56.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.795+0000 7fb40e7fc640 1 -- 172.21.15.5:0/2042825947 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3fc003520 con 0x7fb4100707e0 2023-11-24T06:16:56.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.795+0000 7fb40f7fe640 1 -- 172.21.15.5:0/2042825947 >> v1:172.21.15.6:6789/0 conn(0x7fb4101449d0 legacy=0x7fb4100700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:56.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.795+0000 7fb40f7fe640 1 -- 172.21.15.5:0/2042825947 >> v1:172.21.15.5:6789/0 conn(0x7fb4100707e0 legacy=0x7fb41006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:56.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.795+0000 7fb40f7fe640 1 -- 172.21.15.5:0/2042825947 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41006d600 con 0x7fb410146370 2023-11-24T06:16:56.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.797+0000 7fb40f7fe640 1 --2- 172.21.15.5:0/2042825947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb410146370 0x7fb41006c9e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb408009d90 tx=0x7fb40802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a97a19ac78716c65 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:56.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.797+0000 7fb40e7fc640 1 -- 172.21.15.5:0/2042825947 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40802cd00 con 0x7fb410146370 2023-11-24T06:16:56.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.797+0000 7fb40e7fc640 1 -- 172.21.15.5:0/2042825947 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40802ce60 con 0x7fb410146370 2023-11-24T06:16:56.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.797+0000 7fb40e7fc640 1 -- 172.21.15.5:0/2042825947 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408034660 con 0x7fb410146370 2023-11-24T06:16:56.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.798+0000 7fb40ffff640 1 -- 172.21.15.5:0/2042825947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb410146370 msgr2=0x7fb41006c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:56.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.798+0000 7fb40ffff640 1 --2- 172.21.15.5:0/2042825947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb410146370 0x7fb41006c9e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb408009d90 tx=0x7fb40802bf50 comp rx=0 tx=0).stop 2023-11-24T06:16:56.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.798+0000 7fb40ffff640 1 -- 172.21.15.5:0/2042825947 shutdown_connections 2023-11-24T06:16:56.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.798+0000 7fb40ffff640 1 --2- 172.21.15.5:0/2042825947 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb410146370 0x7fb41006c9e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.798+0000 7fb40ffff640 1 -- 172.21.15.5:0/2042825947 >> 172.21.15.5:0/2042825947 conn(0x7fb4100b0840 msgr2=0x7fb4100b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:56.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.798+0000 7fb40ffff640 1 -- 172.21.15.5:0/2042825947 shutdown_connections 2023-11-24T06:16:56.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.798+0000 7fb40ffff640 1 -- 172.21.15.5:0/2042825947 wait complete. 2023-11-24T06:16:56.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.800+0000 7fb40ffff640 1 Processor -- start 2023-11-24T06:16:56.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.800+0000 7fb40ffff640 1 -- start start 2023-11-24T06:16:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.801+0000 7fb40ffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4100707e0 0x7fb410156c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.801+0000 7fb40ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4101449d0 0x7fb410153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.801+0000 7fb40ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb410146370 0x7fb4100b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.801+0000 7fb40ffff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4101562b0 con 0x7fb4101449d0 2023-11-24T06:16:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.801+0000 7fb40ffff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4101567e0 con 0x7fb410146370 2023-11-24T06:16:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.801+0000 7fb40ffff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100b17f0 con 0x7fb4100707e0 2023-11-24T06:16:56.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.801+0000 7fb407fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4101449d0 0x7fb410153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:56.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.801+0000 7fb407fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4101449d0 0x7fb410153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42918/0 (socket says 172.21.15.5:42918) 2023-11-24T06:16:56.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.801+0000 7fb407fff640 1 -- 172.21.15.5:0/1800336677 learned_addr learned my addr 172.21.15.5:0/1800336677 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:56.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.801+0000 7fb40effd640 1 --2- 172.21.15.5:0/1800336677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4100707e0 0x7fb410156c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:56.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.801+0000 7fb407fff640 1 -- 172.21.15.5:0/1800336677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4100707e0 msgr2=0x7fb410156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:56.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.802+0000 7fb40f7fe640 1 --2- 172.21.15.5:0/1800336677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb410146370 0x7fb4100b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:56.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.802+0000 7fb407fff640 1 --2- 172.21.15.5:0/1800336677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4100707e0 0x7fb410156c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.802+0000 7fb407fff640 1 -- 172.21.15.5:0/1800336677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb410146370 msgr2=0x7fb4100b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.802+0000 7fb407fff640 1 --2- 172.21.15.5:0/1800336677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb410146370 0x7fb4100b0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.802+0000 7fb407fff640 1 -- 172.21.15.5:0/1800336677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb408009a70 con 0x7fb4101449d0 2023-11-24T06:16:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.802+0000 7fb407fff640 1 --2- 172.21.15.5:0/1800336677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4101449d0 0x7fb410153110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800c500 tx=0x7fb3f800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.802+0000 7fb4077fe640 1 -- 172.21.15.5:0/1800336677 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8005ce0 con 0x7fb4101449d0 2023-11-24T06:16:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.802+0000 7fb4077fe640 1 -- 172.21.15.5:0/1800336677 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f8005e40 con 0x7fb4101449d0 2023-11-24T06:16:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.802+0000 7fb4077fe640 1 -- 172.21.15.5:0/1800336677 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8007330 con 0x7fb4101449d0 2023-11-24T06:16:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.802+0000 7fb40ffff640 1 -- 172.21.15.5:0/1800336677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4100b1ad0 con 0x7fb4101449d0 2023-11-24T06:16:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.802+0000 7fb40ffff640 1 -- 172.21.15.5:0/1800336677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb410153a90 con 0x7fb4101449d0 2023-11-24T06:16:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.804+0000 7fb40ffff640 1 -- 172.21.15.5:0/1800336677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4101569b0 con 0x7fb4101449d0 2023-11-24T06:16:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.805+0000 7fb4077fe640 1 -- 172.21.15.5:0/1800336677 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb3f8007550 con 0x7fb4101449d0 2023-11-24T06:16:56.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.806+0000 7fb4077fe640 1 --2- 172.21.15.5:0/1800336677 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb3f006ae40 0x7fb3f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:56.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.806+0000 7fb4077fe640 1 -- 172.21.15.5:0/1800336677 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb3f8088c80 con 0x7fb4101449d0 2023-11-24T06:16:56.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.806+0000 7fb40effd640 1 --2- 172.21.15.5:0/1800336677 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb3f006ae40 0x7fb3f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:56.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.806+0000 7fb40effd640 1 --2- 172.21.15.5:0/1800336677 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb3f006ae40 0x7fb3f006d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb400000f60 tx=0x7fb400009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:56.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:56.811+0000 7fb4077fe640 1 -- 172.21.15.5:0/1800336677 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3f8053df0 con 0x7fb4101449d0 2023-11-24T06:16:57.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.019+0000 7fb40ffff640 1 -- 172.21.15.5:0/1800336677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb41004e840 con 0x7fb4101449d0 2023-11-24T06:16:57.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.019+0000 7fb4077fe640 1 -- 172.21.15.5:0/1800336677 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb3f8017020 con 0x7fb4101449d0 2023-11-24T06:16:57.022 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2023-11-24T06:16:57.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.022+0000 7fb40ffff640 1 -- 172.21.15.5:0/1800336677 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb3f006ae40 msgr2=0x7fb3f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:57.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.022+0000 7fb40ffff640 1 --2- 172.21.15.5:0/1800336677 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb3f006ae40 0x7fb3f006d300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb400000f60 tx=0x7fb400009040 comp rx=0 tx=0).stop 2023-11-24T06:16:57.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.022+0000 7fb40ffff640 1 -- 172.21.15.5:0/1800336677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4101449d0 msgr2=0x7fb410153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:57.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.022+0000 7fb40ffff640 1 --2- 172.21.15.5:0/1800336677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4101449d0 0x7fb410153110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800c500 tx=0x7fb3f800c9c0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.022+0000 7fb40ffff640 1 -- 172.21.15.5:0/1800336677 shutdown_connections 2023-11-24T06:16:57.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.022+0000 7fb40ffff640 1 --2- 172.21.15.5:0/1800336677 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb3f006ae40 0x7fb3f006d300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.022+0000 7fb40ffff640 1 --2- 172.21.15.5:0/1800336677 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb410146370 0x7fb4100b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.022+0000 7fb40ffff640 1 --2- 172.21.15.5:0/1800336677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4101449d0 0x7fb410153110 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.022+0000 7fb40ffff640 1 --2- 172.21.15.5:0/1800336677 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4100707e0 0x7fb410156c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.022+0000 7fb40ffff640 1 -- 172.21.15.5:0/1800336677 >> 172.21.15.5:0/1800336677 conn(0x7fb4100b0840 msgr2=0x7fb410161c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:57.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.022+0000 7fb40ffff640 1 -- 172.21.15.5:0/1800336677 shutdown_connections 2023-11-24T06:16:57.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.022+0000 7fb40ffff640 1 -- 172.21.15.5:0/1800336677 wait complete. 2023-11-24T06:16:57.047 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:16:57.047 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:16:57.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:16:57.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.344+0000 7f96dfe5c640 1 Processor -- start 2023-11-24T06:16:57.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.344+0000 7f96dfe5c640 1 -- start start 2023-11-24T06:16:57.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.345+0000 7f96dfe5c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96d815b2c0 0x7f96d80cfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:57.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.345+0000 7f96dfe5c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d80d00f0 con 0x7f96d806ff20 2023-11-24T06:16:57.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.345+0000 7f96dfe5c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d80d03d0 con 0x7f96d80b46a0 2023-11-24T06:16:57.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.345+0000 7f96dfe5c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d80d06b0 con 0x7f96d815b2c0 2023-11-24T06:16:57.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.345+0000 7f96ddbf8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f96d815b2c0 0x7f96d80cfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:57.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.346+0000 7f96de3f9640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f96d806ff20 0x7f96d80d1190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46280/0 (socket says 172.21.15.5:46280) 2023-11-24T06:16:57.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.346+0000 7f96de3f9640 1 -- 172.21.15.5:0/1670189444 learned_addr learned my addr 172.21.15.5:0/1670189444 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:57.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.346+0000 7f96ddbf8640 1 -- 172.21.15.5:0/1670189444 >> v1:172.21.15.6:6789/0 conn(0x7f96d80b46a0 legacy=0x7f96d815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:57.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.346+0000 7f96ddbf8640 1 -- 172.21.15.5:0/1670189444 >> v1:172.21.15.5:6789/0 conn(0x7f96d806ff20 legacy=0x7f96d80d1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:57.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.346+0000 7f96ddbf8640 1 -- 172.21.15.5:0/1670189444 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d80d0990 con 0x7f96d815b2c0 2023-11-24T06:16:57.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.347+0000 7f96ddbf8640 1 --2- 172.21.15.5:0/1670189444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d815b2c0 0x7f96d80cfaf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f96d40097a0 tx=0x7f96d402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd53106c3b6e1ad9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:57.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.347+0000 7f96dcbf6640 1 -- 172.21.15.5:0/1670189444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d402e070 con 0x7f96d815b2c0 2023-11-24T06:16:57.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.347+0000 7f96dcbf6640 1 -- 172.21.15.5:0/1670189444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d40315a0 con 0x7f96d815b2c0 2023-11-24T06:16:57.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.348+0000 7f96dcbf6640 1 -- 172.21.15.5:0/1670189444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4031930 con 0x7f96d815b2c0 2023-11-24T06:16:57.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.348+0000 7f96dfe5c640 1 -- 172.21.15.5:0/1670189444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d815b2c0 msgr2=0x7f96d80cfaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:57.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.348+0000 7f96dfe5c640 1 --2- 172.21.15.5:0/1670189444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d815b2c0 0x7f96d80cfaf0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f96d40097a0 tx=0x7f96d402b840 comp rx=0 tx=0).stop 2023-11-24T06:16:57.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.348+0000 7f96dfe5c640 1 -- 172.21.15.5:0/1670189444 shutdown_connections 2023-11-24T06:16:57.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.348+0000 7f96dfe5c640 1 --2- 172.21.15.5:0/1670189444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d815b2c0 0x7f96d80cfaf0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.348+0000 7f96dfe5c640 1 -- 172.21.15.5:0/1670189444 >> 172.21.15.5:0/1670189444 conn(0x7f96d80b2e00 msgr2=0x7f96d80b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:57.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.348+0000 7f96dfe5c640 1 -- 172.21.15.5:0/1670189444 shutdown_connections 2023-11-24T06:16:57.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.348+0000 7f96dfe5c640 1 -- 172.21.15.5:0/1670189444 wait complete. 2023-11-24T06:16:57.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.350+0000 7f96dfe5c640 1 Processor -- start 2023-11-24T06:16:57.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.350+0000 7f96dfe5c640 1 -- start start 2023-11-24T06:16:57.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.351+0000 7f96dfe5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d806ff20 0x7f96d8154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:57.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.351+0000 7f96dfe5c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d80b46a0 0x7f96d8159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:57.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.351+0000 7f96dfe5c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d815b2c0 0x7f96d8148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:57.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.351+0000 7f96dfe5c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d8158c80 con 0x7f96d806ff20 2023-11-24T06:16:57.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.351+0000 7f96dfe5c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d8158f60 con 0x7f96d80b46a0 2023-11-24T06:16:57.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.351+0000 7f96dfe5c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d8159240 con 0x7f96d815b2c0 2023-11-24T06:16:57.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.351+0000 7f96ddbf8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d806ff20 0x7f96d8154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:57.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.351+0000 7f96ddbf8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d806ff20 0x7f96d8154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42938/0 (socket says 172.21.15.5:42938) 2023-11-24T06:16:57.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.351+0000 7f96ddbf8640 1 -- 172.21.15.5:0/964548223 learned_addr learned my addr 172.21.15.5:0/964548223 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:57.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.352+0000 7f96de3f9640 1 --2- 172.21.15.5:0/964548223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d815b2c0 0x7f96d8148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:57.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.352+0000 7f96ddbf8640 1 -- 172.21.15.5:0/964548223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d815b2c0 msgr2=0x7f96d8148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:57.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.352+0000 7f96ddbf8640 1 --2- 172.21.15.5:0/964548223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d815b2c0 0x7f96d8148aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.352+0000 7f96dd3f7640 1 --2- 172.21.15.5:0/964548223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d80b46a0 0x7f96d8159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:57.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.352+0000 7f96ddbf8640 1 -- 172.21.15.5:0/964548223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d80b46a0 msgr2=0x7f96d8159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:57.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.352+0000 7f96ddbf8640 1 --2- 172.21.15.5:0/964548223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d80b46a0 0x7f96d8159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.352+0000 7f96ddbf8640 1 -- 172.21.15.5:0/964548223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d4009400 con 0x7f96d806ff20 2023-11-24T06:16:57.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.352+0000 7f96ddbf8640 1 --2- 172.21.15.5:0/964548223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d806ff20 0x7f96d8154c80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f96d4034040 tx=0x7f96d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:57.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.353+0000 7f96ce7fc640 1 -- 172.21.15.5:0/964548223 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4032900 con 0x7f96d806ff20 2023-11-24T06:16:57.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.353+0000 7f96ce7fc640 1 -- 172.21.15.5:0/964548223 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d4032a80 con 0x7f96d806ff20 2023-11-24T06:16:57.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.353+0000 7f96ce7fc640 1 -- 172.21.15.5:0/964548223 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4032e30 con 0x7f96d806ff20 2023-11-24T06:16:57.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.353+0000 7f96dfe5c640 1 -- 172.21.15.5:0/964548223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d8159520 con 0x7f96d806ff20 2023-11-24T06:16:57.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.353+0000 7f96dfe5c640 1 -- 172.21.15.5:0/964548223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d8148fe0 con 0x7f96d806ff20 2023-11-24T06:16:57.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.354+0000 7f96dfe5c640 1 -- 172.21.15.5:0/964548223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d8074770 con 0x7f96d806ff20 2023-11-24T06:16:57.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.355+0000 7f96ce7fc640 1 -- 172.21.15.5:0/964548223 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f96d403b3f0 con 0x7f96d806ff20 2023-11-24T06:16:57.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.356+0000 7f96ce7fc640 1 --2- 172.21.15.5:0/964548223 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f96bc06ae40 0x7f96bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:57.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.356+0000 7f96ce7fc640 1 -- 172.21.15.5:0/964548223 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f96d40ab840 con 0x7f96d806ff20 2023-11-24T06:16:57.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.356+0000 7f96dd3f7640 1 --2- 172.21.15.5:0/964548223 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f96bc06ae40 0x7f96bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:57.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.357+0000 7f96dd3f7640 1 --2- 172.21.15.5:0/964548223 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f96bc06ae40 0x7f96bc06d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f96d8156a00 tx=0x7f96c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:57.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.360+0000 7f96ce7fc640 1 -- 172.21.15.5:0/964548223 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f96d4076730 con 0x7f96d806ff20 2023-11-24T06:16:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.555+0000 7f96dfe5c640 1 -- 172.21.15.5:0/964548223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96d80b3c00 con 0x7f96d806ff20 2023-11-24T06:16:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.558+0000 7f96ce7fc640 1 -- 172.21.15.5:0/964548223 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16658 (secure 0 0 0) 0x7f96d4079d50 con 0x7f96d806ff20 2023-11-24T06:16:57.562 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:16:57.562 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:16:33.958286+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:16:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.561+0000 7f96dfe5c640 1 -- 172.21.15.5:0/964548223 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f96bc06ae40 msgr2=0x7f96bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.561+0000 7f96dfe5c640 1 --2- 172.21.15.5:0/964548223 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f96bc06ae40 0x7f96bc06d300 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f96d8156a00 tx=0x7f96c8009040 comp rx=0 tx=0).stop 2023-11-24T06:16:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.561+0000 7f96dfe5c640 1 -- 172.21.15.5:0/964548223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d806ff20 msgr2=0x7f96d8154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.561+0000 7f96dfe5c640 1 --2- 172.21.15.5:0/964548223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d806ff20 0x7f96d8154c80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f96d4034040 tx=0x7f96d40046c0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.561+0000 7f96dfe5c640 1 -- 172.21.15.5:0/964548223 shutdown_connections 2023-11-24T06:16:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.561+0000 7f96dfe5c640 1 --2- 172.21.15.5:0/964548223 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f96bc06ae40 0x7f96bc06d300 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.561+0000 7f96dfe5c640 1 --2- 172.21.15.5:0/964548223 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f96d815b2c0 0x7f96d8148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.561+0000 7f96dfe5c640 1 --2- 172.21.15.5:0/964548223 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f96d80b46a0 0x7f96d8159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.561+0000 7f96dfe5c640 1 --2- 172.21.15.5:0/964548223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96d806ff20 0x7f96d8154c80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.561+0000 7f96dfe5c640 1 -- 172.21.15.5:0/964548223 >> 172.21.15.5:0/964548223 conn(0x7f96d80b2e00 msgr2=0x7f96d80d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.561+0000 7f96dfe5c640 1 -- 172.21.15.5:0/964548223 shutdown_connections 2023-11-24T06:16:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.561+0000 7f96dfe5c640 1 -- 172.21.15.5:0/964548223 wait complete. 2023-11-24T06:16:57.585 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:16:57.586 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:16:57.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.803+0000 7f318e1fa640 1 Processor -- start 2023-11-24T06:16:57.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.803+0000 7f318e1fa640 1 -- start start 2023-11-24T06:16:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.805+0000 7f318e1fa640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f31880ad450 0x7f3188154280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.805+0000 7f318e1fa640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3188070850 0x7f3188071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.805+0000 7f318e1fa640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31880ce650 con 0x7f3188153ea0 2023-11-24T06:16:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.805+0000 7f318e1fa640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31880ad8e0 con 0x7f31880ad450 2023-11-24T06:16:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.805+0000 7f318e1fa640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31881547c0 con 0x7f3188070850 2023-11-24T06:16:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.805+0000 7f318c9f7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f31880ad450 0x7f3188154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.806+0000 7f318c9f7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f31880ad450 0x7f3188154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42942/0 (socket says 172.21.15.5:42942) 2023-11-24T06:16:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.806+0000 7f318c9f7640 1 -- 172.21.15.5:0/1104471302 learned_addr learned my addr 172.21.15.5:0/1104471302 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.806+0000 7f318d9f9640 1 --2- 172.21.15.5:0/1104471302 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3188070850 0x7f3188071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.806+0000 7f318c9f7640 1 -- 172.21.15.5:0/1104471302 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3188070850 msgr2=0x7f3188071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.806+0000 7f318c9f7640 1 --2- 172.21.15.5:0/1104471302 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3188070850 0x7f3188071350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.806+0000 7f318c9f7640 1 -- 172.21.15.5:0/1104471302 >> v1:172.21.15.6:6789/0 conn(0x7f3188153ea0 legacy=0x7f318815d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.806+0000 7f318c9f7640 1 -- 172.21.15.5:0/1104471302 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318815e0a0 con 0x7f31880ad450 2023-11-24T06:16:57.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.807+0000 7f318c9f7640 1 --2- 172.21.15.5:0/1104471302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31880ad450 0x7f3188154280 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f31780098a0 tx=0x7f317802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e93317928dee13b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:57.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.808+0000 7f317ffff640 1 -- 172.21.15.5:0/1104471302 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178030840 con 0x7f31880ad450 2023-11-24T06:16:57.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.808+0000 7f317ffff640 1 -- 172.21.15.5:0/1104471302 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31780309a0 con 0x7f31880ad450 2023-11-24T06:16:57.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.808+0000 7f317ffff640 1 -- 172.21.15.5:0/1104471302 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178030d30 con 0x7f31880ad450 2023-11-24T06:16:57.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.808+0000 7f318e1fa640 1 -- 172.21.15.5:0/1104471302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31880ad450 msgr2=0x7f3188154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:57.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.808+0000 7f318e1fa640 1 --2- 172.21.15.5:0/1104471302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31880ad450 0x7f3188154280 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f31780098a0 tx=0x7f317802b980 comp rx=0 tx=0).stop 2023-11-24T06:16:57.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.808+0000 7f318e1fa640 1 -- 172.21.15.5:0/1104471302 shutdown_connections 2023-11-24T06:16:57.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.808+0000 7f318e1fa640 1 --2- 172.21.15.5:0/1104471302 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3188070850 0x7f3188071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.808+0000 7f318e1fa640 1 --2- 172.21.15.5:0/1104471302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31880ad450 0x7f3188154280 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.808+0000 7f318e1fa640 1 -- 172.21.15.5:0/1104471302 >> 172.21.15.5:0/1104471302 conn(0x7f31880cec10 msgr2=0x7f31880cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:57.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.808+0000 7f318e1fa640 1 -- 172.21.15.5:0/1104471302 shutdown_connections 2023-11-24T06:16:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.808+0000 7f318e1fa640 1 -- 172.21.15.5:0/1104471302 wait complete. 2023-11-24T06:16:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.810+0000 7f318e1fa640 1 Processor -- start 2023-11-24T06:16:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.810+0000 7f318e1fa640 1 -- start start 2023-11-24T06:16:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.811+0000 7f318e1fa640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3188070850 0x7f318813c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.811+0000 7f318e1fa640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31880ad450 0x7f318814d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.811+0000 7f318e1fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3188153ea0 0x7f31881f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.811+0000 7f318e1fa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318813cc50 con 0x7f3188153ea0 2023-11-24T06:16:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.811+0000 7f318e1fa640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3188156b20 con 0x7f31880ad450 2023-11-24T06:16:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.811+0000 7f318e1fa640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3188155f10 con 0x7f3188070850 2023-11-24T06:16:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.812+0000 7f318d9f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3188153ea0 0x7f31881f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.812+0000 7f318d9f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3188153ea0 0x7f31881f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42958/0 (socket says 172.21.15.5:42958) 2023-11-24T06:16:57.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.812+0000 7f318d9f9640 1 -- 172.21.15.5:0/911399225 learned_addr learned my addr 172.21.15.5:0/911399225 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.812+0000 7f318d1f8640 1 --2- 172.21.15.5:0/911399225 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3188070850 0x7f318813c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.812+0000 7f318d9f9640 1 -- 172.21.15.5:0/911399225 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3188070850 msgr2=0x7f318813c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.812+0000 7f318c9f7640 1 --2- 172.21.15.5:0/911399225 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31880ad450 0x7f318814d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.812+0000 7f318d9f9640 1 --2- 172.21.15.5:0/911399225 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3188070850 0x7f318813c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.812+0000 7f318d9f9640 1 -- 172.21.15.5:0/911399225 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31880ad450 msgr2=0x7f318814d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.812+0000 7f318d9f9640 1 --2- 172.21.15.5:0/911399225 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31880ad450 0x7f318814d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.812+0000 7f318d9f9640 1 -- 172.21.15.5:0/911399225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31780094d0 con 0x7f3188153ea0 2023-11-24T06:16:57.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.813+0000 7f318d9f9640 1 --2- 172.21.15.5:0/911399225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3188153ea0 0x7f31881f2fb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f318000efc0 tx=0x7f318000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.813+0000 7f317dffb640 1 -- 172.21.15.5:0/911399225 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180014480 con 0x7f3188153ea0 2023-11-24T06:16:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.813+0000 7f317dffb640 1 -- 172.21.15.5:0/911399225 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318000f040 con 0x7f3188153ea0 2023-11-24T06:16:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.813+0000 7f317dffb640 1 -- 172.21.15.5:0/911399225 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3180014820 con 0x7f3188153ea0 2023-11-24T06:16:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.813+0000 7f318e1fa640 1 -- 172.21.15.5:0/911399225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31881561f0 con 0x7f3188153ea0 2023-11-24T06:16:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.813+0000 7f318e1fa640 1 -- 172.21.15.5:0/911399225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31881567b0 con 0x7f3188153ea0 2023-11-24T06:16:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.814+0000 7f318e1fa640 1 -- 172.21.15.5:0/911399225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3188156c90 con 0x7f3188153ea0 2023-11-24T06:16:57.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.815+0000 7f317dffb640 1 -- 172.21.15.5:0/911399225 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3180014980 con 0x7f3188153ea0 2023-11-24T06:16:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.816+0000 7f317dffb640 1 --2- 172.21.15.5:0/911399225 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f316c06ae40 0x7f316c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.816+0000 7f317dffb640 1 -- 172.21.15.5:0/911399225 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f318008a380 con 0x7f3188153ea0 2023-11-24T06:16:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.816+0000 7f318d1f8640 1 --2- 172.21.15.5:0/911399225 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f316c06ae40 0x7f316c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:57.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.817+0000 7f318d1f8640 1 --2- 172.21.15.5:0/911399225 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f316c06ae40 0x7f316c06d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3184000f60 tx=0x7f3184009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:57.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:57.820+0000 7f317dffb640 1 -- 172.21.15.5:0/911399225 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3180055310 con 0x7f3188153ea0 2023-11-24T06:16:58.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.014+0000 7f318e1fa640 1 -- 172.21.15.5:0/911399225 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f318804e840 con 0x7f3188153ea0 2023-11-24T06:16:58.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.014+0000 7f317dffb640 1 -- 172.21.15.5:0/911399225 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31800180a0 con 0x7f3188153ea0 2023-11-24T06:16:58.017 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:16:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.017+0000 7f318e1fa640 1 -- 172.21.15.5:0/911399225 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f316c06ae40 msgr2=0x7f316c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.017+0000 7f318e1fa640 1 --2- 172.21.15.5:0/911399225 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f316c06ae40 0x7f316c06d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3184000f60 tx=0x7f3184009040 comp rx=0 tx=0).stop 2023-11-24T06:16:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.017+0000 7f318e1fa640 1 -- 172.21.15.5:0/911399225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3188153ea0 msgr2=0x7f31881f2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.017+0000 7f318e1fa640 1 --2- 172.21.15.5:0/911399225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3188153ea0 0x7f31881f2fb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f318000efc0 tx=0x7f318000c490 comp rx=0 tx=0).stop 2023-11-24T06:16:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.017+0000 7f318e1fa640 1 -- 172.21.15.5:0/911399225 shutdown_connections 2023-11-24T06:16:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.018+0000 7f318e1fa640 1 --2- 172.21.15.5:0/911399225 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f316c06ae40 0x7f316c06d300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.018+0000 7f318e1fa640 1 --2- 172.21.15.5:0/911399225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3188153ea0 0x7f31881f2fb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.018+0000 7f318e1fa640 1 --2- 172.21.15.5:0/911399225 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31880ad450 0x7f318814d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.018+0000 7f318e1fa640 1 --2- 172.21.15.5:0/911399225 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3188070850 0x7f318813c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.018+0000 7f318e1fa640 1 -- 172.21.15.5:0/911399225 >> 172.21.15.5:0/911399225 conn(0x7f31880cec10 msgr2=0x7f318806c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.018+0000 7f318e1fa640 1 -- 172.21.15.5:0/911399225 shutdown_connections 2023-11-24T06:16:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.018+0000 7f318e1fa640 1 -- 172.21.15.5:0/911399225 wait complete. 2023-11-24T06:16:58.040 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:16:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.258+0000 7f10b1747640 1 Processor -- start 2023-11-24T06:16:58.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.258+0000 7f10b1747640 1 -- start start 2023-11-24T06:16:58.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.260+0000 7f10b1747640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f10ac0d1d90 0x7f10ac0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.260+0000 7f10b1747640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f10ac0d0fa0 0x7f10ac0b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.260+0000 7f10b1747640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ac0c6bf0 con 0x7f10ac15d3b0 2023-11-24T06:16:58.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.260+0000 7f10b1747640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ac0d13f0 con 0x7f10ac0d1d90 2023-11-24T06:16:58.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.260+0000 7f10b1747640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ac0cedc0 con 0x7f10ac0d0fa0 2023-11-24T06:16:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.260+0000 7f10abfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f10ac15d3b0 0x7f10ac0d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46284/0 (socket says 172.21.15.5:46284) 2023-11-24T06:16:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.260+0000 7f10abfff640 1 -- 172.21.15.5:0/2392517456 learned_addr learned my addr 172.21.15.5:0/2392517456 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.260+0000 7f10ab7fe640 1 --2- 172.21.15.5:0/2392517456 >> v2:172.21.15.5:3300/0 conn(0x7f10ac0d1d90 0x7f10ac0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.260+0000 7f10b0f46640 1 --2- 172.21.15.5:0/2392517456 >> v2:172.21.15.6:3300/0 conn(0x7f10ac0d0fa0 0x7f10ac0b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.261+0000 7f10aaffd640 1 -- 172.21.15.5:0/2392517456 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 344120259 0 0) 0x7f10ac0c6bf0 con 0x7f10ac15d3b0 2023-11-24T06:16:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.261+0000 7f10aaffd640 1 -- 172.21.15.5:0/2392517456 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1098003520 con 0x7f10ac15d3b0 2023-11-24T06:16:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.261+0000 7f10ab7fe640 1 -- 172.21.15.5:0/2392517456 >> v2:172.21.15.6:3300/0 conn(0x7f10ac0d0fa0 msgr2=0x7f10ac0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.261+0000 7f10ab7fe640 1 --2- 172.21.15.5:0/2392517456 >> v2:172.21.15.6:3300/0 conn(0x7f10ac0d0fa0 0x7f10ac0b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.261+0000 7f10ab7fe640 1 -- 172.21.15.5:0/2392517456 >> v1:172.21.15.5:6789/0 conn(0x7f10ac15d3b0 legacy=0x7f10ac0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:58.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.261+0000 7f10ab7fe640 1 -- 172.21.15.5:0/2392517456 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac0c6e40 con 0x7f10ac0d1d90 2023-11-24T06:16:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.262+0000 7f10ab7fe640 1 --2- 172.21.15.5:0/2392517456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0d1d90 0x7f10ac0ce6c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f10ac072b10 tx=0x7f109402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a100fd940c13dc2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.262+0000 7f10aaffd640 1 -- 172.21.15.5:0/2392517456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109402e070 con 0x7f10ac0d1d90 2023-11-24T06:16:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.262+0000 7f10aaffd640 1 -- 172.21.15.5:0/2392517456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10940315a0 con 0x7f10ac0d1d90 2023-11-24T06:16:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.262+0000 7f10aaffd640 1 -- 172.21.15.5:0/2392517456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094031930 con 0x7f10ac0d1d90 2023-11-24T06:16:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.263+0000 7f10b1747640 1 -- 172.21.15.5:0/2392517456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0d1d90 msgr2=0x7f10ac0ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.263+0000 7f10b1747640 1 --2- 172.21.15.5:0/2392517456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0d1d90 0x7f10ac0ce6c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f10ac072b10 tx=0x7f109402b840 comp rx=0 tx=0).stop 2023-11-24T06:16:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.263+0000 7f10b1747640 1 -- 172.21.15.5:0/2392517456 shutdown_connections 2023-11-24T06:16:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.263+0000 7f10b1747640 1 --2- 172.21.15.5:0/2392517456 >> v2:172.21.15.6:3300/0 conn(0x7f10ac0d0fa0 0x7f10ac0b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.263+0000 7f10b1747640 1 --2- 172.21.15.5:0/2392517456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0d1d90 0x7f10ac0ce6c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.263+0000 7f10b1747640 1 -- 172.21.15.5:0/2392517456 >> 172.21.15.5:0/2392517456 conn(0x7f10ac0d30a0 msgr2=0x7f10ac0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.263+0000 7f10b1747640 1 -- 172.21.15.5:0/2392517456 shutdown_connections 2023-11-24T06:16:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.263+0000 7f10b1747640 1 -- 172.21.15.5:0/2392517456 wait complete. 2023-11-24T06:16:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.265+0000 7f10b1747640 1 Processor -- start 2023-11-24T06:16:58.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.265+0000 7f10b1747640 1 -- start start 2023-11-24T06:16:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.266+0000 7f10b1747640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0d0fa0 0x7f10ac151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.266+0000 7f10b1747640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10ac0d1d90 0x7f10ac0b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.266+0000 7f10b1747640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f10ac15d3b0 0x7f10ac0caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.266+0000 7f10b1747640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac1523f0 con 0x7f10ac0d0fa0 2023-11-24T06:16:58.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.266+0000 7f10b1747640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0b6e50 con 0x7f10ac0d1d90 2023-11-24T06:16:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.266+0000 7f10b1747640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0b7130 con 0x7f10ac15d3b0 2023-11-24T06:16:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.266+0000 7f10abfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0d0fa0 0x7f10ac151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.266+0000 7f10abfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0d0fa0 0x7f10ac151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42966/0 (socket says 172.21.15.5:42966) 2023-11-24T06:16:58.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.266+0000 7f10abfff640 1 -- 172.21.15.5:0/4147446486 learned_addr learned my addr 172.21.15.5:0/4147446486 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:58.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.267+0000 7f10abfff640 1 -- 172.21.15.5:0/4147446486 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f10ac15d3b0 msgr2=0x7f10ac0caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:58.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.267+0000 7f10abfff640 1 --2- 172.21.15.5:0/4147446486 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f10ac15d3b0 0x7f10ac0caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.267+0000 7f10abfff640 1 -- 172.21.15.5:0/4147446486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10ac0d1d90 msgr2=0x7f10ac0b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.267+0000 7f10abfff640 1 --2- 172.21.15.5:0/4147446486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10ac0d1d90 0x7f10ac0b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.267+0000 7f10abfff640 1 -- 172.21.15.5:0/4147446486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1094009400 con 0x7f10ac0d0fa0 2023-11-24T06:16:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.267+0000 7f10abfff640 1 --2- 172.21.15.5:0/4147446486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0d0fa0 0x7f10ac151eb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f10a400cbc0 tx=0x7f10a400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.267+0000 7f10a8ff9640 1 -- 172.21.15.5:0/4147446486 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a400ad40 con 0x7f10ac0d0fa0 2023-11-24T06:16:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.267+0000 7f10a8ff9640 1 -- 172.21.15.5:0/4147446486 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a4012400 con 0x7f10ac0d0fa0 2023-11-24T06:16:58.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.267+0000 7f10a8ff9640 1 -- 172.21.15.5:0/4147446486 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a400aea0 con 0x7f10ac0d0fa0 2023-11-24T06:16:58.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.267+0000 7f10b1747640 1 -- 172.21.15.5:0/4147446486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ac0b7410 con 0x7f10ac0d0fa0 2023-11-24T06:16:58.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.267+0000 7f10b1747640 1 -- 172.21.15.5:0/4147446486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ac0b79d0 con 0x7f10ac0d0fa0 2023-11-24T06:16:58.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.269+0000 7f10b1747640 1 -- 172.21.15.5:0/4147446486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ac04e840 con 0x7f10ac0d0fa0 2023-11-24T06:16:58.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.270+0000 7f10a8ff9640 1 -- 172.21.15.5:0/4147446486 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f10a4007070 con 0x7f10ac0d0fa0 2023-11-24T06:16:58.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.270+0000 7f10a8ff9640 1 --2- 172.21.15.5:0/4147446486 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f108c06ae40 0x7f108c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.271+0000 7f10a8ff9640 1 -- 172.21.15.5:0/4147446486 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f10a4089720 con 0x7f10ac0d0fa0 2023-11-24T06:16:58.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.271+0000 7f10ab7fe640 1 --2- 172.21.15.5:0/4147446486 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f108c06ae40 0x7f108c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:58.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.271+0000 7f10ab7fe640 1 --2- 172.21.15.5:0/4147446486 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f108c06ae40 0x7f108c06d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1094036000 tx=0x7f1094030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:58.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.275+0000 7f10a8ff9640 1 -- 172.21.15.5:0/4147446486 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f10a40581d0 con 0x7f10ac0d0fa0 2023-11-24T06:16:58.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.467+0000 7f10b1747640 1 -- 172.21.15.5:0/4147446486 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10ac0b7bc0 con 0x7f10ac0d0fa0 2023-11-24T06:16:58.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.468+0000 7f10a8ff9640 1 -- 172.21.15.5:0/4147446486 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f10a4057d60 con 0x7f10ac0d0fa0 2023-11-24T06:16:58.471 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:16:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.471+0000 7f10b1747640 1 -- 172.21.15.5:0/4147446486 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f108c06ae40 msgr2=0x7f108c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.471+0000 7f10b1747640 1 --2- 172.21.15.5:0/4147446486 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f108c06ae40 0x7f108c06d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1094036000 tx=0x7f1094030000 comp rx=0 tx=0).stop 2023-11-24T06:16:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.471+0000 7f10b1747640 1 -- 172.21.15.5:0/4147446486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0d0fa0 msgr2=0x7f10ac151eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.471+0000 7f10b1747640 1 --2- 172.21.15.5:0/4147446486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0d0fa0 0x7f10ac151eb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f10a400cbc0 tx=0x7f10a400a430 comp rx=0 tx=0).stop 2023-11-24T06:16:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.471+0000 7f10b1747640 1 -- 172.21.15.5:0/4147446486 shutdown_connections 2023-11-24T06:16:58.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.471+0000 7f10b1747640 1 --2- 172.21.15.5:0/4147446486 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f108c06ae40 0x7f108c06d300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.471+0000 7f10b1747640 1 --2- 172.21.15.5:0/4147446486 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f10ac15d3b0 0x7f10ac0caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.471+0000 7f10b1747640 1 --2- 172.21.15.5:0/4147446486 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f10ac0d1d90 0x7f10ac0b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.471+0000 7f10b1747640 1 --2- 172.21.15.5:0/4147446486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10ac0d0fa0 0x7f10ac151eb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.471+0000 7f10b1747640 1 -- 172.21.15.5:0/4147446486 >> 172.21.15.5:0/4147446486 conn(0x7f10ac0d30a0 msgr2=0x7f10ac0c8600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.471+0000 7f10b1747640 1 -- 172.21.15.5:0/4147446486 shutdown_connections 2023-11-24T06:16:58.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.471+0000 7f10b1747640 1 -- 172.21.15.5:0/4147446486 wait complete. 2023-11-24T06:16:58.493 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:16:58.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.703+0000 7fa586b06640 1 Processor -- start 2023-11-24T06:16:58.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.703+0000 7fa586b06640 1 -- start start 2023-11-24T06:16:58.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.705+0000 7fa586b06640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa5800cfa30 0x7fa5800cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.705+0000 7fa586b06640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5800b33d0 0x7fa5800b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.705+0000 7fa586b06640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5800b2980 con 0x7fa5800d1fd0 2023-11-24T06:16:58.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.705+0000 7fa586b06640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5800cf030 con 0x7fa5800cfa30 2023-11-24T06:16:58.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.705+0000 7fa586b06640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5800d0350 con 0x7fa5800b33d0 2023-11-24T06:16:58.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.705+0000 7fa585303640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa5800d1fd0 0x7fa5800ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46296/0 (socket says 172.21.15.5:46296) 2023-11-24T06:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.705+0000 7fa585303640 1 -- 172.21.15.5:0/977847144 learned_addr learned my addr 172.21.15.5:0/977847144 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.706+0000 7fa585b04640 1 --2- 172.21.15.5:0/977847144 >> v2:172.21.15.6:3300/0 conn(0x7fa5800cfa30 0x7fa5800cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.706+0000 7fa586305640 1 --2- 172.21.15.5:0/977847144 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5800b33d0 0x7fa5800b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.706+0000 7fa584b02640 1 -- 172.21.15.5:0/977847144 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173400442 0 0) 0x7fa5800b2980 con 0x7fa5800d1fd0 2023-11-24T06:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.706+0000 7fa584b02640 1 -- 172.21.15.5:0/977847144 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa56c003520 con 0x7fa5800d1fd0 2023-11-24T06:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.706+0000 7fa584b02640 1 -- 172.21.15.5:0/977847144 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 577660111 0 0) 0x7fa56c003520 con 0x7fa5800d1fd0 2023-11-24T06:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.706+0000 7fa584b02640 1 -- 172.21.15.5:0/977847144 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5800b33d0 msgr2=0x7fa5800b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.706+0000 7fa584b02640 1 --2- 172.21.15.5:0/977847144 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5800b33d0 0x7fa5800b3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.706+0000 7fa584b02640 1 -- 172.21.15.5:0/977847144 >> v2:172.21.15.6:3300/0 conn(0x7fa5800cfa30 msgr2=0x7fa5800cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.706+0000 7fa584b02640 1 --2- 172.21.15.5:0/977847144 >> v2:172.21.15.6:3300/0 conn(0x7fa5800cfa30 0x7fa5800cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.706+0000 7fa584b02640 1 -- 172.21.15.5:0/977847144 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5800cf220 con 0x7fa5800d1fd0 2023-11-24T06:16:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.706+0000 7fa584b02640 1 -- 172.21.15.5:0/977847144 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fa570001670 con 0x7fa5800d1fd0 2023-11-24T06:16:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.706+0000 7fa584b02640 1 -- 172.21.15.5:0/977847144 >> v1:172.21.15.5:6789/0 conn(0x7fa5800d1fd0 legacy=0x7fa5800ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.706+0000 7fa584b02640 1 --2- 172.21.15.5:0/977847144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa56c0039e0 0x7fa56c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.706+0000 7fa584b02640 1 --2- 172.21.15.5:0/977847144 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5800b33d0 0x7fa56c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.707+0000 7fa584b02640 1 --2- 172.21.15.5:0/977847144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa56c004d00 0x7fa56c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.707+0000 7fa584b02640 1 -- 172.21.15.5:0/977847144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5800d0350 con 0x7fa56c004d00 2023-11-24T06:16:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.707+0000 7fa584b02640 1 -- 172.21.15.5:0/977847144 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c00b980 con 0x7fa56c0039e0 2023-11-24T06:16:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.707+0000 7fa584b02640 1 -- 172.21.15.5:0/977847144 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c00bac0 con 0x7fa5800b33d0 2023-11-24T06:16:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.707+0000 7fa586305640 1 --2- 172.21.15.5:0/977847144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa56c004d00 0x7fa56c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.707+0000 7fa585303640 1 --2- 172.21.15.5:0/977847144 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5800b33d0 0x7fa56c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.707+0000 7fa585b04640 1 --2- 172.21.15.5:0/977847144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa56c0039e0 0x7fa56c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.707+0000 7fa586305640 1 -- 172.21.15.5:0/977847144 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5800b33d0 msgr2=0x7fa56c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.707+0000 7fa586305640 1 --2- 172.21.15.5:0/977847144 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5800b33d0 0x7fa56c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.707+0000 7fa586305640 1 -- 172.21.15.5:0/977847144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa56c0039e0 msgr2=0x7fa56c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.707+0000 7fa586305640 1 --2- 172.21.15.5:0/977847144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa56c0039e0 0x7fa56c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.707+0000 7fa586305640 1 -- 172.21.15.5:0/977847144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c003520 con 0x7fa56c004d00 2023-11-24T06:16:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.707+0000 7fa585303640 1 --2- 172.21.15.5:0/977847144 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5800b33d0 0x7fa56c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.707+0000 7fa585b04640 1 --2- 172.21.15.5:0/977847144 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa56c0039e0 0x7fa56c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.708+0000 7fa586305640 1 --2- 172.21.15.5:0/977847144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa56c004d00 0x7fa56c005180 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa578009ac0 tx=0x7fa56c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751772b8587d532f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.709+0000 7fa584b02640 1 -- 172.21.15.5:0/977847144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57802c6b0 con 0x7fa56c004d00 2023-11-24T06:16:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.709+0000 7fa584b02640 1 -- 172.21.15.5:0/977847144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57802c810 con 0x7fa56c004d00 2023-11-24T06:16:58.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.709+0000 7fa584b02640 1 -- 172.21.15.5:0/977847144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57802cba0 con 0x7fa56c004d00 2023-11-24T06:16:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.709+0000 7fa586b06640 1 -- 172.21.15.5:0/977847144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa56c004d00 msgr2=0x7fa56c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.709+0000 7fa586b06640 1 --2- 172.21.15.5:0/977847144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa56c004d00 0x7fa56c005180 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa578009ac0 tx=0x7fa56c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.709+0000 7fa586305640 1 -- 172.21.15.5:0/977847144 reap_dead start 2023-11-24T06:16:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.709+0000 7fa586b06640 1 -- 172.21.15.5:0/977847144 shutdown_connections 2023-11-24T06:16:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.709+0000 7fa586b06640 1 -- 172.21.15.5:0/977847144 >> 172.21.15.5:0/977847144 conn(0x7fa5800d3020 msgr2=0x7fa5800d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.710+0000 7fa586b06640 1 -- 172.21.15.5:0/977847144 shutdown_connections 2023-11-24T06:16:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.710+0000 7fa586b06640 1 -- 172.21.15.5:0/977847144 wait complete. 2023-11-24T06:16:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.711+0000 7fa586b06640 1 Processor -- start 2023-11-24T06:16:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.711+0000 7fa586b06640 1 -- start start 2023-11-24T06:16:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.712+0000 7fa586b06640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa56c004d00 0x7fa580152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.712+0000 7fa586b06640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa58015e420 0x7fa5800c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.712+0000 7fa586b06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5800cc230 0x7fa5800ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.712+0000 7fa586b06640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa580152c80 con 0x7fa5800cc230 2023-11-24T06:16:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.712+0000 7fa586b06640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5800c80c0 con 0x7fa56c004d00 2023-11-24T06:16:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.712+0000 7fa586b06640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5800ccff0 con 0x7fa58015e420 2023-11-24T06:16:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.713+0000 7fa586305640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5800cc230 0x7fa5800ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.713+0000 7fa586305640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5800cc230 0x7fa5800ca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42974/0 (socket says 172.21.15.5:42974) 2023-11-24T06:16:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.713+0000 7fa586305640 1 -- 172.21.15.5:0/4269099812 learned_addr learned my addr 172.21.15.5:0/4269099812 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.713+0000 7fa585303640 1 --2- 172.21.15.5:0/4269099812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa58015e420 0x7fa5800c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.713+0000 7fa585b04640 1 --2- 172.21.15.5:0/4269099812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa56c004d00 0x7fa580152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.713+0000 7fa586305640 1 -- 172.21.15.5:0/4269099812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa58015e420 msgr2=0x7fa5800c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.713+0000 7fa586305640 1 --2- 172.21.15.5:0/4269099812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa58015e420 0x7fa5800c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.713+0000 7fa586305640 1 -- 172.21.15.5:0/4269099812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa56c004d00 msgr2=0x7fa580152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.713+0000 7fa586305640 1 --2- 172.21.15.5:0/4269099812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa56c004d00 0x7fa580152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.713+0000 7fa586305640 1 -- 172.21.15.5:0/4269099812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5800cca30 con 0x7fa5800cc230 2023-11-24T06:16:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.714+0000 7fa586305640 1 --2- 172.21.15.5:0/4269099812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5800cc230 0x7fa5800ca0f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa578037040 tx=0x7fa578004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.714+0000 7fa5767fc640 1 -- 172.21.15.5:0/4269099812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57802cd90 con 0x7fa5800cc230 2023-11-24T06:16:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.714+0000 7fa5767fc640 1 -- 172.21.15.5:0/4269099812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa578003610 con 0x7fa5800cc230 2023-11-24T06:16:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.714+0000 7fa5767fc640 1 -- 172.21.15.5:0/4269099812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5780039c0 con 0x7fa5800cc230 2023-11-24T06:16:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.714+0000 7fa586b06640 1 -- 172.21.15.5:0/4269099812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5800ccd10 con 0x7fa5800cc230 2023-11-24T06:16:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.714+0000 7fa586b06640 1 -- 172.21.15.5:0/4269099812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58015ef80 con 0x7fa5800cc230 2023-11-24T06:16:58.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.716+0000 7fa586b06640 1 -- 172.21.15.5:0/4269099812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5800d0a40 con 0x7fa5800cc230 2023-11-24T06:16:58.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.717+0000 7fa5767fc640 1 -- 172.21.15.5:0/4269099812 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa578003b20 con 0x7fa5800cc230 2023-11-24T06:16:58.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.718+0000 7fa5767fc640 1 --2- 172.21.15.5:0/4269099812 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa56406ae40 0x7fa56406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:58.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.718+0000 7fa5767fc640 1 -- 172.21.15.5:0/4269099812 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa578000e40 con 0x7fa5800cc230 2023-11-24T06:16:58.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.718+0000 7fa585b04640 1 --2- 172.21.15.5:0/4269099812 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa56406ae40 0x7fa56406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:58.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.719+0000 7fa585b04640 1 --2- 172.21.15.5:0/4269099812 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa56406ae40 0x7fa56406d300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa57c003be0 tx=0x7fa57c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:58.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.722+0000 7fa5767fc640 1 -- 172.21.15.5:0/4269099812 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa578075b70 con 0x7fa5800cc230 2023-11-24T06:16:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.918+0000 7fa586b06640 1 -- 172.21.15.5:0/4269099812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa580066d20 con 0x7fa5800cc230 2023-11-24T06:16:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.919+0000 7fa5767fc640 1 -- 172.21.15.5:0/4269099812 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa578079190 con 0x7fa5800cc230 2023-11-24T06:16:58.922 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:16:58.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.921+0000 7fa586b06640 1 -- 172.21.15.5:0/4269099812 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa56406ae40 msgr2=0x7fa56406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:58.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.921+0000 7fa586b06640 1 --2- 172.21.15.5:0/4269099812 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa56406ae40 0x7fa56406d300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa57c003be0 tx=0x7fa57c00b040 comp rx=0 tx=0).stop 2023-11-24T06:16:58.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.921+0000 7fa586b06640 1 -- 172.21.15.5:0/4269099812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5800cc230 msgr2=0x7fa5800ca0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:58.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.921+0000 7fa586b06640 1 --2- 172.21.15.5:0/4269099812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5800cc230 0x7fa5800ca0f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa578037040 tx=0x7fa578004730 comp rx=0 tx=0).stop 2023-11-24T06:16:58.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.921+0000 7fa586b06640 1 -- 172.21.15.5:0/4269099812 shutdown_connections 2023-11-24T06:16:58.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.921+0000 7fa586b06640 1 --2- 172.21.15.5:0/4269099812 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa56406ae40 0x7fa56406d300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.921+0000 7fa586b06640 1 --2- 172.21.15.5:0/4269099812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa58015e420 0x7fa5800c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.921+0000 7fa586b06640 1 --2- 172.21.15.5:0/4269099812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5800cc230 0x7fa5800ca0f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.921+0000 7fa586b06640 1 --2- 172.21.15.5:0/4269099812 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa56c004d00 0x7fa580152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:58.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.922+0000 7fa586b06640 1 -- 172.21.15.5:0/4269099812 >> 172.21.15.5:0/4269099812 conn(0x7fa5800d3020 msgr2=0x7fa5800b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:58.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.922+0000 7fa586b06640 1 -- 172.21.15.5:0/4269099812 shutdown_connections 2023-11-24T06:16:58.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:58.922+0000 7fa586b06640 1 -- 172.21.15.5:0/4269099812 wait complete. 2023-11-24T06:16:58.944 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:16:59.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.163+0000 7efe75d2c640 1 Processor -- start 2023-11-24T06:16:59.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.164+0000 7efe75d2c640 1 -- start start 2023-11-24T06:16:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.165+0000 7efe75d2c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efe7006d520 0x7efe7006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.165+0000 7efe75d2c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7efe700b2080 0x7efe700b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.165+0000 7efe75d2c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe70070dc0 con 0x7efe700acb50 2023-11-24T06:16:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.165+0000 7efe75d2c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe700720e0 con 0x7efe7006d520 2023-11-24T06:16:59.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.165+0000 7efe75d2c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe70161d00 con 0x7efe700b2080 2023-11-24T06:16:59.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.166+0000 7efe6f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efe7006d520 0x7efe7006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:59.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.166+0000 7efe6f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efe7006d520 0x7efe7006d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42990/0 (socket says 172.21.15.5:42990) 2023-11-24T06:16:59.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.166+0000 7efe6f7fe640 1 -- 172.21.15.5:0/3735965699 learned_addr learned my addr 172.21.15.5:0/3735965699 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:59.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.166+0000 7efe6ffff640 1 --2- 172.21.15.5:0/3735965699 >> v2:172.21.15.6:3300/0 conn(0x7efe700b2080 0x7efe700b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.166+0000 7efe6f7fe640 1 -- 172.21.15.5:0/3735965699 >> v2:172.21.15.6:3300/0 conn(0x7efe700b2080 msgr2=0x7efe700b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.166+0000 7efe6f7fe640 1 --2- 172.21.15.5:0/3735965699 >> v2:172.21.15.6:3300/0 conn(0x7efe700b2080 0x7efe700b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.166+0000 7efe6f7fe640 1 -- 172.21.15.5:0/3735965699 >> v1:172.21.15.6:6789/0 conn(0x7efe700acb50 legacy=0x7efe700719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.166+0000 7efe6f7fe640 1 -- 172.21.15.5:0/3735965699 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7006de40 con 0x7efe7006d520 2023-11-24T06:16:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.167+0000 7efe6f7fe640 1 --2- 172.21.15.5:0/3735965699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7006d520 0x7efe7006d900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efe700b19e0 tx=0x7efe6002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490ce50136eb06d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:59.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.168+0000 7efe6effd640 1 -- 172.21.15.5:0/3735965699 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60031870 con 0x7efe7006d520 2023-11-24T06:16:59.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.168+0000 7efe6effd640 1 -- 172.21.15.5:0/3735965699 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe600319d0 con 0x7efe7006d520 2023-11-24T06:16:59.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.168+0000 7efe6effd640 1 -- 172.21.15.5:0/3735965699 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60031d60 con 0x7efe7006d520 2023-11-24T06:16:59.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.168+0000 7efe75d2c640 1 -- 172.21.15.5:0/3735965699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7006d520 msgr2=0x7efe7006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:59.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.168+0000 7efe75d2c640 1 --2- 172.21.15.5:0/3735965699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7006d520 0x7efe7006d900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efe700b19e0 tx=0x7efe6002b910 comp rx=0 tx=0).stop 2023-11-24T06:16:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.168+0000 7efe75d2c640 1 -- 172.21.15.5:0/3735965699 shutdown_connections 2023-11-24T06:16:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.168+0000 7efe75d2c640 1 --2- 172.21.15.5:0/3735965699 >> v2:172.21.15.6:3300/0 conn(0x7efe700b2080 0x7efe700b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.168+0000 7efe75d2c640 1 --2- 172.21.15.5:0/3735965699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7006d520 0x7efe7006d900 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.168+0000 7efe75d2c640 1 -- 172.21.15.5:0/3735965699 >> 172.21.15.5:0/3735965699 conn(0x7efe700b0d80 msgr2=0x7efe700b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.168+0000 7efe75d2c640 1 -- 172.21.15.5:0/3735965699 shutdown_connections 2023-11-24T06:16:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.168+0000 7efe75d2c640 1 -- 172.21.15.5:0/3735965699 wait complete. 2023-11-24T06:16:59.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.170+0000 7efe75d2c640 1 Processor -- start 2023-11-24T06:16:59.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.170+0000 7efe75d2c640 1 -- start start 2023-11-24T06:16:59.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.171+0000 7efe75d2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7006d520 0x7efe7014caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:59.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.171+0000 7efe75d2c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe700acb50 0x7efe7014b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.171+0000 7efe75d2c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe700b2080 0x7efe7014ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.171+0000 7efe75d2c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7014dd70 con 0x7efe7006d520 2023-11-24T06:16:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.171+0000 7efe75d2c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7014d160 con 0x7efe700acb50 2023-11-24T06:16:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.171+0000 7efe75d2c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7014d440 con 0x7efe700b2080 2023-11-24T06:16:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.172+0000 7efe6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7006d520 0x7efe7014caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.172+0000 7efe6f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7006d520 0x7efe7014caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42992/0 (socket says 172.21.15.5:42992) 2023-11-24T06:16:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.172+0000 7efe6f7fe640 1 -- 172.21.15.5:0/1806709714 learned_addr learned my addr 172.21.15.5:0/1806709714 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.172+0000 7efe67fff640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe700acb50 0x7efe7014b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.172+0000 7efe6ffff640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe700b2080 0x7efe7014ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:59.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.172+0000 7efe6f7fe640 1 -- 172.21.15.5:0/1806709714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe700b2080 msgr2=0x7efe7014ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.172+0000 7efe6f7fe640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe700b2080 0x7efe7014ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.172+0000 7efe6f7fe640 1 -- 172.21.15.5:0/1806709714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe700acb50 msgr2=0x7efe7014b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.172+0000 7efe6f7fe640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe700acb50 0x7efe7014b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.172+0000 7efe6f7fe640 1 -- 172.21.15.5:0/1806709714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe600094d0 con 0x7efe7006d520 2023-11-24T06:16:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.172+0000 7efe67fff640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe700acb50 0x7efe7014b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.172+0000 7efe6ffff640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe700b2080 0x7efe7014ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:16:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.173+0000 7efe6f7fe640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7006d520 0x7efe7014caa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efe60000c00 tx=0x7efe600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.173+0000 7efe6cff9640 1 -- 172.21.15.5:0/1806709714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6003abe0 con 0x7efe7006d520 2023-11-24T06:16:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.173+0000 7efe6cff9640 1 -- 172.21.15.5:0/1806709714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6003ad40 con 0x7efe7006d520 2023-11-24T06:16:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.173+0000 7efe6cff9640 1 -- 172.21.15.5:0/1806709714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe60044610 con 0x7efe7006d520 2023-11-24T06:16:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.173+0000 7efe75d2c640 1 -- 172.21.15.5:0/1806709714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7014d720 con 0x7efe7006d520 2023-11-24T06:16:59.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.173+0000 7efe75d2c640 1 -- 172.21.15.5:0/1806709714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe700afce0 con 0x7efe7006d520 2023-11-24T06:16:59.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.174+0000 7efe75d2c640 1 -- 172.21.15.5:0/1806709714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe7014d580 con 0x7efe7006d520 2023-11-24T06:16:59.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.175+0000 7efe6cff9640 1 -- 172.21.15.5:0/1806709714 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efe6004d450 con 0x7efe7006d520 2023-11-24T06:16:59.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.176+0000 7efe6cff9640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efe5006ae40 0x7efe5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:59.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.176+0000 7efe6cff9640 1 -- 172.21.15.5:0/1806709714 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7efe6002d040 con 0x7efe7006d520 2023-11-24T06:16:59.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.176+0000 7efe67fff640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efe5006ae40 0x7efe5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:59.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.177+0000 7efe67fff640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efe5006ae40 0x7efe5006d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efe70071b20 tx=0x7efe58008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.181+0000 7efe6cff9640 1 -- 172.21.15.5:0/1806709714 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efe7014d580 con 0x7efe7006d520 2023-11-24T06:16:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.388+0000 7efe75d2c640 1 -- 172.21.15.5:0/1806709714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7efe7014d580 con 0x7efe7006d520 2023-11-24T06:16:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.388+0000 7efe6cff9640 1 -- 172.21.15.5:0/1806709714 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7efe7014d580 con 0x7efe7006d520 2023-11-24T06:16:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2023-11-24T06:16:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.391+0000 7efe75d2c640 1 -- 172.21.15.5:0/1806709714 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efe5006ae40 msgr2=0x7efe5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.391+0000 7efe75d2c640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efe5006ae40 0x7efe5006d300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efe70071b20 tx=0x7efe58008040 comp rx=0 tx=0).stop 2023-11-24T06:16:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.391+0000 7efe75d2c640 1 -- 172.21.15.5:0/1806709714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7006d520 msgr2=0x7efe7014caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.391+0000 7efe75d2c640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7006d520 0x7efe7014caa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7efe60000c00 tx=0x7efe600046c0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.391+0000 7efe75d2c640 1 -- 172.21.15.5:0/1806709714 shutdown_connections 2023-11-24T06:16:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.391+0000 7efe75d2c640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efe5006ae40 0x7efe5006d300 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.391+0000 7efe75d2c640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efe700b2080 0x7efe7014ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.391+0000 7efe75d2c640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efe700acb50 0x7efe7014b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.391+0000 7efe75d2c640 1 --2- 172.21.15.5:0/1806709714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe7006d520 0x7efe7014caa0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.391+0000 7efe75d2c640 1 -- 172.21.15.5:0/1806709714 >> 172.21.15.5:0/1806709714 conn(0x7efe700b0d80 msgr2=0x7efe701605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.391+0000 7efe75d2c640 1 -- 172.21.15.5:0/1806709714 shutdown_connections 2023-11-24T06:16:59.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.391+0000 7efe75d2c640 1 -- 172.21.15.5:0/1806709714 wait complete. 2023-11-24T06:16:59.413 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi006.front.sepia.ceph.com... 2023-11-24T06:16:59.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:59.414 DEBUG:teuthology.orchestra.run.smithi006:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-24T06:16:59.514 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:16:59.550 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T06:16:59.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:16:59.550 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:16:59.614 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T06:16:59.615 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:16:59.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.850+0000 7f85b72e1640 1 Processor -- start 2023-11-24T06:16:59.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.851+0000 7f85b72e1640 1 -- start start 2023-11-24T06:16:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.852+0000 7f85b72e1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f85b0157770 0x7f85b006cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.852+0000 7f85b72e1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b015df80 0x7f85b0070720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:59.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.852+0000 7f85b72e1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b0161670 con 0x7f85b006d080 2023-11-24T06:16:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.852+0000 7f85b72e1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b015e410 con 0x7f85b0157770 2023-11-24T06:16:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.852+0000 7f85b72e1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b006d500 con 0x7f85b015df80 2023-11-24T06:16:59.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.853+0000 7f85b62df640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f85b0157770 0x7f85b006cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.853+0000 7f85b6ae0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b015df80 0x7f85b0070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.853+0000 7f85b5ade640 1 --1- >> v1:172.21.15.6:6789/0 conn(0x7f85b006d080 0x7f85b015d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.6:6789/0 says I am v1:172.21.15.5:48000/0 (socket says 172.21.15.5:48000) 2023-11-24T06:16:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.853+0000 7f85b5ade640 1 -- 172.21.15.5:0/2359022120 learned_addr learned my addr 172.21.15.5:0/2359022120 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:16:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.853+0000 7f85b62df640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f85b0157770 0x7f85b006cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:56858/0 (socket says 172.21.15.5:56858) 2023-11-24T06:16:59.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.854+0000 7f85b62df640 1 -- 172.21.15.5:0/2359022120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b015df80 msgr2=0x7f85b0070720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:16:59.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.854+0000 7f85b62df640 1 --2- 172.21.15.5:0/2359022120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b015df80 0x7f85b0070720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.854+0000 7f85b62df640 1 -- 172.21.15.5:0/2359022120 >> v1:172.21.15.6:6789/0 conn(0x7f85b006d080 legacy=0x7f85b015d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:59.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.854+0000 7f85b62df640 1 -- 172.21.15.5:0/2359022120 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b015e610 con 0x7f85b0157770 2023-11-24T06:16:59.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.855+0000 7f85b62df640 1 --2- 172.21.15.5:0/2359022120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b0157770 0x7f85b006cb40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0097a0 tx=0x7f85ac02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81417fdce15f1b5d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:59.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.856+0000 7f85b52dd640 1 -- 172.21.15.5:0/2359022120 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac02d040 con 0x7f85b0157770 2023-11-24T06:16:59.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.857+0000 7f85b52dd640 1 -- 172.21.15.5:0/2359022120 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac031810 con 0x7f85b0157770 2023-11-24T06:16:59.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.857+0000 7f85b52dd640 1 -- 172.21.15.5:0/2359022120 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac031ba0 con 0x7f85b0157770 2023-11-24T06:16:59.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.857+0000 7f85b72e1640 1 -- 172.21.15.5:0/2359022120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b0157770 msgr2=0x7f85b006cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:59.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.857+0000 7f85b72e1640 1 --2- 172.21.15.5:0/2359022120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b0157770 0x7f85b006cb40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0097a0 tx=0x7f85ac02b8b0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.857+0000 7f85b72e1640 1 -- 172.21.15.5:0/2359022120 shutdown_connections 2023-11-24T06:16:59.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.857+0000 7f85b72e1640 1 --2- 172.21.15.5:0/2359022120 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b015df80 0x7f85b0070720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.857+0000 7f85b72e1640 1 --2- 172.21.15.5:0/2359022120 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b0157770 0x7f85b006cb40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.857+0000 7f85b72e1640 1 -- 172.21.15.5:0/2359022120 >> 172.21.15.5:0/2359022120 conn(0x7f85b006a860 msgr2=0x7f85b006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:16:59.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.857+0000 7f85b72e1640 1 -- 172.21.15.5:0/2359022120 shutdown_connections 2023-11-24T06:16:59.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.857+0000 7f85b72e1640 1 -- 172.21.15.5:0/2359022120 wait complete. 2023-11-24T06:16:59.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.860+0000 7f85b72e1640 1 Processor -- start 2023-11-24T06:16:59.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.860+0000 7f85b72e1640 1 -- start start 2023-11-24T06:16:59.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.861+0000 7f85b72e1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b006d080 0x7f85b0154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:59.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.861+0000 7f85b72e1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b0157770 0x7f85b015fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:59.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.861+0000 7f85b72e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b015df80 0x7f85b00cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:59.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.861+0000 7f85b72e1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b01f6e60 con 0x7f85b015df80 2023-11-24T06:16:59.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.861+0000 7f85b72e1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b0147f90 con 0x7f85b0157770 2023-11-24T06:16:59.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.861+0000 7f85b72e1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b01f5f00 con 0x7f85b006d080 2023-11-24T06:16:59.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.861+0000 7f85b6ae0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b015df80 0x7f85b00cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:59.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.862+0000 7f85b6ae0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b015df80 0x7f85b00cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43002/0 (socket says 172.21.15.5:43002) 2023-11-24T06:16:59.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.862+0000 7f85b6ae0640 1 -- 172.21.15.5:0/3251673364 learned_addr learned my addr 172.21.15.5:0/3251673364 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:16:59.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.862+0000 7f85b62df640 1 --2- 172.21.15.5:0/3251673364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b006d080 0x7f85b0154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:59.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.862+0000 7f85b5ade640 1 --2- 172.21.15.5:0/3251673364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b0157770 0x7f85b015fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:59.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.862+0000 7f85b6ae0640 1 -- 172.21.15.5:0/3251673364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b006d080 msgr2=0x7f85b0154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:59.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.862+0000 7f85b6ae0640 1 --2- 172.21.15.5:0/3251673364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b006d080 0x7f85b0154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.862+0000 7f85b6ae0640 1 -- 172.21.15.5:0/3251673364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b0157770 msgr2=0x7f85b015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:16:59.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.862+0000 7f85b6ae0640 1 --2- 172.21.15.5:0/3251673364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b0157770 0x7f85b015fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:16:59.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.862+0000 7f85b6ae0640 1 -- 172.21.15.5:0/3251673364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac009400 con 0x7f85b015df80 2023-11-24T06:16:59.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.862+0000 7f85b6ae0640 1 --2- 172.21.15.5:0/3251673364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b015df80 0x7f85b00cce10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f85a800efc0 tx=0x7f85a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:59.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.863+0000 7f85a6ffd640 1 -- 172.21.15.5:0/3251673364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a8009ba0 con 0x7f85b015df80 2023-11-24T06:16:59.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.863+0000 7f85a6ffd640 1 -- 172.21.15.5:0/3251673364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a800f040 con 0x7f85b015df80 2023-11-24T06:16:59.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.863+0000 7f85b72e1640 1 -- 172.21.15.5:0/3251673364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b01f61e0 con 0x7f85b015df80 2023-11-24T06:16:59.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.863+0000 7f85a6ffd640 1 -- 172.21.15.5:0/3251673364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a80146c0 con 0x7f85b015df80 2023-11-24T06:16:59.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.863+0000 7f85b72e1640 1 -- 172.21.15.5:0/3251673364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b01f67a0 con 0x7f85b015df80 2023-11-24T06:16:59.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.864+0000 7f85b72e1640 1 -- 172.21.15.5:0/3251673364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b004e840 con 0x7f85b015df80 2023-11-24T06:16:59.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.866+0000 7f85a6ffd640 1 -- 172.21.15.5:0/3251673364 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85a8014820 con 0x7f85b015df80 2023-11-24T06:16:59.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.866+0000 7f85a6ffd640 1 --2- 172.21.15.5:0/3251673364 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f859406ae40 0x7f859406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:16:59.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.867+0000 7f85a6ffd640 1 -- 172.21.15.5:0/3251673364 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f85a808a4e0 con 0x7f85b015df80 2023-11-24T06:16:59.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.867+0000 7f85b62df640 1 --2- 172.21.15.5:0/3251673364 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f859406ae40 0x7f859406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:16:59.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.868+0000 7f85b62df640 1 --2- 172.21.15.5:0/3251673364 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f859406ae40 0x7f859406d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f85ac036000 tx=0x7f85ac03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:16:59.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:16:59.871+0000 7f85a6ffd640 1 -- 172.21.15.5:0/3251673364 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85a80554f0 con 0x7f85b015df80 2023-11-24T06:17:00.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.102+0000 7f85b72e1640 1 -- 172.21.15.5:0/3251673364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85b0070500 con 0x7f85b015df80 2023-11-24T06:17:00.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.103+0000 7f85a6ffd640 1 -- 172.21.15.5:0/3251673364 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4895 (secure 0 0 0) 0x7f85a8058b10 con 0x7f85b015df80 2023-11-24T06:17:00.107 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:00.107 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4009460946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4009460946},{"type":"v1","addr":"172.21.15.5:6837","nonce":4009460946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2496926427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2496926427},{"type":"v1","addr":"172.21.15.5:6836","nonce":2496926427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3829754769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3829754769},{"type":"v1","addr":"172.21.15.6:6835","nonce":3829754769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:15:51.611541+0000","modified":"2023-11-24T06:15:54.664234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6833/2760317954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2760317954},{"type":"v1","addr":"172.21.15.6:6833","nonce":2760317954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:17:00.107 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2023-11-24T06:17:00.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.106+0000 7f85b72e1640 1 -- 172.21.15.5:0/3251673364 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f859406ae40 msgr2=0x7f859406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:00.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.106+0000 7f85b72e1640 1 --2- 172.21.15.5:0/3251673364 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f859406ae40 0x7f859406d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f85ac036000 tx=0x7f85ac03c040 comp rx=0 tx=0).stop 2023-11-24T06:17:00.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.106+0000 7f85b72e1640 1 -- 172.21.15.5:0/3251673364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b015df80 msgr2=0x7f85b00cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:00.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.106+0000 7f85b72e1640 1 --2- 172.21.15.5:0/3251673364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b015df80 0x7f85b00cce10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f85a800efc0 tx=0x7f85a800c490 comp rx=0 tx=0).stop 2023-11-24T06:17:00.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.106+0000 7f85b72e1640 1 -- 172.21.15.5:0/3251673364 shutdown_connections 2023-11-24T06:17:00.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.106+0000 7f85b72e1640 1 --2- 172.21.15.5:0/3251673364 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f859406ae40 0x7f859406d300 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.106+0000 7f85b72e1640 1 --2- 172.21.15.5:0/3251673364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85b015df80 0x7f85b00cce10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.106+0000 7f85b72e1640 1 --2- 172.21.15.5:0/3251673364 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f85b0157770 0x7f85b015fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.106+0000 7f85b72e1640 1 --2- 172.21.15.5:0/3251673364 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f85b006d080 0x7f85b0154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.106+0000 7f85b72e1640 1 -- 172.21.15.5:0/3251673364 >> 172.21.15.5:0/3251673364 conn(0x7f85b006a860 msgr2=0x7f85b0155730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:00.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.106+0000 7f85b72e1640 1 -- 172.21.15.5:0/3251673364 shutdown_connections 2023-11-24T06:17:00.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.107+0000 7f85b72e1640 1 -- 172.21.15.5:0/3251673364 wait complete. 2023-11-24T06:17:00.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:17:00.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.353+0000 7fdb15f33640 1 Processor -- start 2023-11-24T06:17:00.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.353+0000 7fdb15f33640 1 -- start start 2023-11-24T06:17:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.355+0000 7fdb15f33640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdb1006dff0 0x7fdb100cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:00.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.355+0000 7fdb15f33640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdb100ce290 0x7fdb1015bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:00.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.355+0000 7fdb15f33640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb1006f1f0 con 0x7fdb10144d80 2023-11-24T06:17:00.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.355+0000 7fdb15f33640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb100ce720 con 0x7fdb1006dff0 2023-11-24T06:17:00.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.355+0000 7fdb15f33640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb1006f4d0 con 0x7fdb100ce290 2023-11-24T06:17:00.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.355+0000 7fdb0effd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdb1006dff0 0x7fdb100cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:00.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.355+0000 7fdb0f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdb10144d80 0x7fdb1006ea10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46302/0 (socket says 172.21.15.5:46302) 2023-11-24T06:17:00.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.355+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/269609982 learned_addr learned my addr 172.21.15.5:0/269609982 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.356+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/269609982 >> v2:172.21.15.6:3300/0 conn(0x7fdb100ce290 0x7fdb1015bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.356+0000 7fdb0e7fc640 1 -- 172.21.15.5:0/269609982 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636838874 0 0) 0x7fdb1006f1f0 con 0x7fdb10144d80 2023-11-24T06:17:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.356+0000 7fdb0e7fc640 1 -- 172.21.15.5:0/269609982 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdafc003520 con 0x7fdb10144d80 2023-11-24T06:17:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.356+0000 7fdb0effd640 1 -- 172.21.15.5:0/269609982 >> v2:172.21.15.6:3300/0 conn(0x7fdb100ce290 msgr2=0x7fdb1015bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.356+0000 7fdb0effd640 1 --2- 172.21.15.5:0/269609982 >> v2:172.21.15.6:3300/0 conn(0x7fdb100ce290 0x7fdb1015bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.356+0000 7fdb0effd640 1 -- 172.21.15.5:0/269609982 >> v1:172.21.15.5:6789/0 conn(0x7fdb10144d80 legacy=0x7fdb1006ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.356+0000 7fdb0effd640 1 -- 172.21.15.5:0/269609982 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb1006f7b0 con 0x7fdb1006dff0 2023-11-24T06:17:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.357+0000 7fdb0effd640 1 --2- 172.21.15.5:0/269609982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1006dff0 0x7fdb100cdd50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdb10073c30 tx=0x7fdaf802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11440c9ad021981 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.358+0000 7fdb0e7fc640 1 -- 172.21.15.5:0/269609982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8031830 con 0x7fdb1006dff0 2023-11-24T06:17:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.358+0000 7fdb0e7fc640 1 -- 172.21.15.5:0/269609982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf8031990 con 0x7fdb1006dff0 2023-11-24T06:17:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.358+0000 7fdb0e7fc640 1 -- 172.21.15.5:0/269609982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf8031d00 con 0x7fdb1006dff0 2023-11-24T06:17:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.358+0000 7fdb15f33640 1 -- 172.21.15.5:0/269609982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1006dff0 msgr2=0x7fdb100cdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.358+0000 7fdb15f33640 1 --2- 172.21.15.5:0/269609982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1006dff0 0x7fdb100cdd50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdb10073c30 tx=0x7fdaf802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.358+0000 7fdb15f33640 1 -- 172.21.15.5:0/269609982 shutdown_connections 2023-11-24T06:17:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.358+0000 7fdb15f33640 1 --2- 172.21.15.5:0/269609982 >> v2:172.21.15.6:3300/0 conn(0x7fdb100ce290 0x7fdb1015bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.358+0000 7fdb15f33640 1 --2- 172.21.15.5:0/269609982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1006dff0 0x7fdb100cdd50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.358+0000 7fdb15f33640 1 -- 172.21.15.5:0/269609982 >> 172.21.15.5:0/269609982 conn(0x7fdb1006d4d0 msgr2=0x7fdb1006d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.358+0000 7fdb15f33640 1 -- 172.21.15.5:0/269609982 shutdown_connections 2023-11-24T06:17:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.358+0000 7fdb15f33640 1 -- 172.21.15.5:0/269609982 wait complete. 2023-11-24T06:17:00.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.360+0000 7fdb15f33640 1 Processor -- start 2023-11-24T06:17:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.361+0000 7fdb15f33640 1 -- start start 2023-11-24T06:17:00.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.361+0000 7fdb15f33640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1006dff0 0x7fdb101f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:00.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.361+0000 7fdb15f33640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb100ce290 0x7fdb101f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:00.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.362+0000 7fdb15f33640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb10144d80 0x7fdb100c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:00.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.362+0000 7fdb15f33640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1012f0c0 con 0x7fdb1006dff0 2023-11-24T06:17:00.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.362+0000 7fdb15f33640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1012f3a0 con 0x7fdb10144d80 2023-11-24T06:17:00.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.362+0000 7fdb15f33640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1012f680 con 0x7fdb100ce290 2023-11-24T06:17:00.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.362+0000 7fdb0f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1006dff0 0x7fdb101f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:00.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.362+0000 7fdb0f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1006dff0 0x7fdb101f1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43026/0 (socket says 172.21.15.5:43026) 2023-11-24T06:17:00.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.362+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/1203668431 learned_addr learned my addr 172.21.15.5:0/1203668431 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:00.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.362+0000 7fdb0ffff640 1 --2- 172.21.15.5:0/1203668431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb10144d80 0x7fdb100c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:00.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.362+0000 7fdb0effd640 1 --2- 172.21.15.5:0/1203668431 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb100ce290 0x7fdb101f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:00.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.362+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/1203668431 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb100ce290 msgr2=0x7fdb101f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:00.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.362+0000 7fdb0f7fe640 1 --2- 172.21.15.5:0/1203668431 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb100ce290 0x7fdb101f3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.362+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/1203668431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb10144d80 msgr2=0x7fdb100c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:00.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.362+0000 7fdb0f7fe640 1 --2- 172.21.15.5:0/1203668431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb10144d80 0x7fdb100c7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.362+0000 7fdb0f7fe640 1 -- 172.21.15.5:0/1203668431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf8009400 con 0x7fdb1006dff0 2023-11-24T06:17:00.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.363+0000 7fdb0f7fe640 1 --2- 172.21.15.5:0/1203668431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1006dff0 0x7fdb101f1950 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000caa0 tx=0x7fdb0000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:00.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.363+0000 7fdb07fff640 1 -- 172.21.15.5:0/1203668431 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb0000ac00 con 0x7fdb1006dff0 2023-11-24T06:17:00.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.363+0000 7fdb07fff640 1 -- 172.21.15.5:0/1203668431 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb0000ad60 con 0x7fdb1006dff0 2023-11-24T06:17:00.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.363+0000 7fdb07fff640 1 -- 172.21.15.5:0/1203668431 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb00012620 con 0x7fdb1006dff0 2023-11-24T06:17:00.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.363+0000 7fdb15f33640 1 -- 172.21.15.5:0/1203668431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1012f960 con 0x7fdb1006dff0 2023-11-24T06:17:00.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.363+0000 7fdb15f33640 1 -- 172.21.15.5:0/1203668431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb101f3ac0 con 0x7fdb1006dff0 2023-11-24T06:17:00.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.364+0000 7fdb15f33640 1 -- 172.21.15.5:0/1203668431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb100cf0c0 con 0x7fdb1006dff0 2023-11-24T06:17:00.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.365+0000 7fdb07fff640 1 -- 172.21.15.5:0/1203668431 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb00007070 con 0x7fdb1006dff0 2023-11-24T06:17:00.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.366+0000 7fdb07fff640 1 --2- 172.21.15.5:0/1203668431 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdaf006ae40 0x7fdaf006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:00.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.367+0000 7fdb07fff640 1 -- 172.21.15.5:0/1203668431 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fdb00089650 con 0x7fdb1006dff0 2023-11-24T06:17:00.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.367+0000 7fdb0effd640 1 --2- 172.21.15.5:0/1203668431 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdaf006ae40 0x7fdaf006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:00.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.367+0000 7fdb0effd640 1 --2- 172.21.15.5:0/1203668431 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdaf006ae40 0x7fdaf006d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8036000 tx=0x7fdaf8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:00.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.371+0000 7fdb07fff640 1 -- 172.21.15.5:0/1203668431 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb00054660 con 0x7fdb1006dff0 2023-11-24T06:17:00.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.570+0000 7fdb15f33640 1 -- 172.21.15.5:0/1203668431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb1012fe80 con 0x7fdb1006dff0 2023-11-24T06:17:00.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.573+0000 7fdb07fff640 1 -- 172.21.15.5:0/1203668431 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16658 (secure 0 0 0) 0x7fdb00017090 con 0x7fdb1006dff0 2023-11-24T06:17:00.576 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:00.577 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:16:33.958286+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:17:00.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.576+0000 7fdb15f33640 1 -- 172.21.15.5:0/1203668431 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdaf006ae40 msgr2=0x7fdaf006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:00.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.576+0000 7fdb15f33640 1 --2- 172.21.15.5:0/1203668431 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdaf006ae40 0x7fdaf006d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8036000 tx=0x7fdaf8008030 comp rx=0 tx=0).stop 2023-11-24T06:17:00.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.576+0000 7fdb15f33640 1 -- 172.21.15.5:0/1203668431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1006dff0 msgr2=0x7fdb101f1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:00.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.576+0000 7fdb15f33640 1 --2- 172.21.15.5:0/1203668431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1006dff0 0x7fdb101f1950 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000caa0 tx=0x7fdb0000cf60 comp rx=0 tx=0).stop 2023-11-24T06:17:00.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.576+0000 7fdb15f33640 1 -- 172.21.15.5:0/1203668431 shutdown_connections 2023-11-24T06:17:00.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.576+0000 7fdb15f33640 1 --2- 172.21.15.5:0/1203668431 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdaf006ae40 0x7fdaf006d300 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.576+0000 7fdb15f33640 1 --2- 172.21.15.5:0/1203668431 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdb100ce290 0x7fdb101f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.576+0000 7fdb15f33640 1 --2- 172.21.15.5:0/1203668431 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdb10144d80 0x7fdb100c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.576+0000 7fdb15f33640 1 --2- 172.21.15.5:0/1203668431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb1006dff0 0x7fdb101f1950 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.576+0000 7fdb15f33640 1 -- 172.21.15.5:0/1203668431 >> 172.21.15.5:0/1203668431 conn(0x7fdb1006d4d0 msgr2=0x7fdb1015d940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:00.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.576+0000 7fdb15f33640 1 -- 172.21.15.5:0/1203668431 shutdown_connections 2023-11-24T06:17:00.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.576+0000 7fdb15f33640 1 -- 172.21.15.5:0/1203668431 wait complete. 2023-11-24T06:17:00.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:15:49.850798+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:15:50.616892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:17:00.600 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:17:00.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.829+0000 7faf72724640 1 Processor -- start 2023-11-24T06:17:00.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.830+0000 7faf72724640 1 -- start start 2023-11-24T06:17:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.831+0000 7faf72724640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faf6c0cd150 0x7faf6c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.831+0000 7faf72724640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7faf6c0702a0 0x7faf6c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.831+0000 7faf72724640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf6c142d50 con 0x7faf6c06ec60 2023-11-24T06:17:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.831+0000 7faf72724640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf6c0cd530 con 0x7faf6c0702a0 2023-11-24T06:17:00.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.831+0000 7faf72724640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c0706c0 con 0x7faf6c0cd150 2023-11-24T06:17:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.832+0000 7faf6bfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faf6c0cd150 0x7faf6c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.832+0000 7faf63fff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7faf6c0702a0 0x7faf6c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.832+0000 7faf6bfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faf6c0cd150 0x7faf6c06fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:34338/0 (socket says 172.21.15.5:34338) 2023-11-24T06:17:00.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.832+0000 7faf6bfff640 1 -- 172.21.15.5:0/3094704100 learned_addr learned my addr 172.21.15.5:0/3094704100 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.833+0000 7faf63fff640 1 -- 172.21.15.5:0/3094704100 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faf6c0cd150 msgr2=0x7faf6c06fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.833+0000 7faf63fff640 1 --2- 172.21.15.5:0/3094704100 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faf6c0cd150 0x7faf6c06fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.833+0000 7faf63fff640 1 -- 172.21.15.5:0/3094704100 >> v1:172.21.15.6:6789/0 conn(0x7faf6c06ec60 legacy=0x7faf6c160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.833+0000 7faf63fff640 1 -- 172.21.15.5:0/3094704100 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c06f430 con 0x7faf6c0702a0 2023-11-24T06:17:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.833+0000 7faf6bfff640 1 --2- 172.21.15.5:0/3094704100 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faf6c0cd150 0x7faf6c06fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:17:00.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.833+0000 7faf6b7fe640 1 -- 172.21.15.5:0/3094704100 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421698483 0 0) 0x7faf6c142d50 con 0x7faf6c06ec60 2023-11-24T06:17:00.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.834+0000 7faf63fff640 1 --2- 172.21.15.5:0/3094704100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c0702a0 0x7faf6c06e720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faf540097a0 tx=0x7faf5402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd4a3aaeb3805989 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:00.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.835+0000 7faf6b7fe640 1 -- 172.21.15.5:0/3094704100 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54031780 con 0x7faf6c0702a0 2023-11-24T06:17:00.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.835+0000 7faf6b7fe640 1 -- 172.21.15.5:0/3094704100 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf540318e0 con 0x7faf6c0702a0 2023-11-24T06:17:00.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.835+0000 7faf6b7fe640 1 -- 172.21.15.5:0/3094704100 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54031c70 con 0x7faf6c0702a0 2023-11-24T06:17:00.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.836+0000 7faf72724640 1 -- 172.21.15.5:0/3094704100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c0702a0 msgr2=0x7faf6c06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:00.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.836+0000 7faf72724640 1 --2- 172.21.15.5:0/3094704100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c0702a0 0x7faf6c06e720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faf540097a0 tx=0x7faf5402b820 comp rx=0 tx=0).stop 2023-11-24T06:17:00.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.836+0000 7faf72724640 1 -- 172.21.15.5:0/3094704100 shutdown_connections 2023-11-24T06:17:00.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.836+0000 7faf72724640 1 --2- 172.21.15.5:0/3094704100 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c0702a0 0x7faf6c06e720 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.836+0000 7faf72724640 1 --2- 172.21.15.5:0/3094704100 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faf6c0cd150 0x7faf6c06fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.836+0000 7faf72724640 1 -- 172.21.15.5:0/3094704100 >> 172.21.15.5:0/3094704100 conn(0x7faf6c0cdb20 msgr2=0x7faf6c0cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:00.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.836+0000 7faf72724640 1 -- 172.21.15.5:0/3094704100 shutdown_connections 2023-11-24T06:17:00.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.836+0000 7faf72724640 1 -- 172.21.15.5:0/3094704100 wait complete. 2023-11-24T06:17:00.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.838+0000 7faf72724640 1 Processor -- start 2023-11-24T06:17:00.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.838+0000 7faf72724640 1 -- start start 2023-11-24T06:17:00.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf72724640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf6c06ec60 0x7faf6c156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:00.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf72724640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c0702a0 0x7faf6c0b5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:00.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf72724640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faf6c0cd150 0x7faf6c0b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:00.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf72724640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c06eb00 con 0x7faf6c06ec60 2023-11-24T06:17:00.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf72724640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c0c8210 con 0x7faf6c0702a0 2023-11-24T06:17:00.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf72724640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6c0c84f0 con 0x7faf6c0cd150 2023-11-24T06:17:00.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf6bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf6c06ec60 0x7faf6c156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:00.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf6bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf6c06ec60 0x7faf6c156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43034/0 (socket says 172.21.15.5:43034) 2023-11-24T06:17:00.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf6bfff640 1 -- 172.21.15.5:0/1194793423 learned_addr learned my addr 172.21.15.5:0/1194793423 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf63fff640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c0702a0 0x7faf6c0b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf70cc1640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faf6c0cd150 0x7faf6c0b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf6bfff640 1 -- 172.21.15.5:0/1194793423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faf6c0cd150 msgr2=0x7faf6c0b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf6bfff640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faf6c0cd150 0x7faf6c0b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf6bfff640 1 -- 172.21.15.5:0/1194793423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c0702a0 msgr2=0x7faf6c0b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf6bfff640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c0702a0 0x7faf6c0b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf6bfff640 1 -- 172.21.15.5:0/1194793423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c0094d0 con 0x7faf6c06ec60 2023-11-24T06:17:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.839+0000 7faf70cc1640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faf6c0cd150 0x7faf6c0b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.840+0000 7faf6bfff640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf6c06ec60 0x7faf6c156e20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00e9d0 tx=0x7faf5c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.840+0000 7faf697fa640 1 -- 172.21.15.5:0/1194793423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00ca60 con 0x7faf6c06ec60 2023-11-24T06:17:00.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.840+0000 7faf697fa640 1 -- 172.21.15.5:0/1194793423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c00cbc0 con 0x7faf6c06ec60 2023-11-24T06:17:00.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.840+0000 7faf697fa640 1 -- 172.21.15.5:0/1194793423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c013470 con 0x7faf6c06ec60 2023-11-24T06:17:00.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.840+0000 7faf72724640 1 -- 172.21.15.5:0/1194793423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf54009400 con 0x7faf6c06ec60 2023-11-24T06:17:00.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.840+0000 7faf72724640 1 -- 172.21.15.5:0/1194793423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6c0c8ab0 con 0x7faf6c06ec60 2023-11-24T06:17:00.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.842+0000 7faf72724640 1 -- 172.21.15.5:0/1194793423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6c0d05e0 con 0x7faf6c06ec60 2023-11-24T06:17:00.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.843+0000 7faf697fa640 1 -- 172.21.15.5:0/1194793423 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf5c0135d0 con 0x7faf6c06ec60 2023-11-24T06:17:00.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.843+0000 7faf697fa640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7faf4c06ae40 0x7faf4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:00.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.844+0000 7faf697fa640 1 -- 172.21.15.5:0/1194793423 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7faf5c0899c0 con 0x7faf6c06ec60 2023-11-24T06:17:00.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.844+0000 7faf63fff640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7faf4c06ae40 0x7faf4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:00.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.845+0000 7faf63fff640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7faf4c06ae40 0x7faf4c06d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faf54036000 tx=0x7faf5403d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:00.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:00.849+0000 7faf697fa640 1 -- 172.21.15.5:0/1194793423 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf5c0549e0 con 0x7faf6c06ec60 2023-11-24T06:17:01.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.043+0000 7faf72724640 1 -- 172.21.15.5:0/1194793423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf6c06fb50 con 0x7faf6c06ec60 2023-11-24T06:17:01.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.044+0000 7faf697fa640 1 -- 172.21.15.5:0/1194793423 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf5c0170a0 con 0x7faf6c06ec60 2023-11-24T06:17:01.047 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:17:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.047+0000 7faf72724640 1 -- 172.21.15.5:0/1194793423 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7faf4c06ae40 msgr2=0x7faf4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.047+0000 7faf72724640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7faf4c06ae40 0x7faf4c06d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faf54036000 tx=0x7faf5403d040 comp rx=0 tx=0).stop 2023-11-24T06:17:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.047+0000 7faf72724640 1 -- 172.21.15.5:0/1194793423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf6c06ec60 msgr2=0x7faf6c156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.047+0000 7faf72724640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf6c06ec60 0x7faf6c156e20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00e9d0 tx=0x7faf5c00ee90 comp rx=0 tx=0).stop 2023-11-24T06:17:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.047+0000 7faf72724640 1 -- 172.21.15.5:0/1194793423 shutdown_connections 2023-11-24T06:17:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.047+0000 7faf72724640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7faf4c06ae40 0x7faf4c06d300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.047+0000 7faf72724640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faf6c0cd150 0x7faf6c0b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.047+0000 7faf72724640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faf6c0702a0 0x7faf6c0b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.047+0000 7faf72724640 1 --2- 172.21.15.5:0/1194793423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf6c06ec60 0x7faf6c156e20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.047+0000 7faf72724640 1 -- 172.21.15.5:0/1194793423 >> 172.21.15.5:0/1194793423 conn(0x7faf6c0cdb20 msgr2=0x7faf6c06c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.047+0000 7faf72724640 1 -- 172.21.15.5:0/1194793423 shutdown_connections 2023-11-24T06:17:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.047+0000 7faf72724640 1 -- 172.21.15.5:0/1194793423 wait complete. 2023-11-24T06:17:01.068 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:17:01.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.285+0000 7f8125dc5640 1 Processor -- start 2023-11-24T06:17:01.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.285+0000 7f8125dc5640 1 -- start start 2023-11-24T06:17:01.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.287+0000 7f8125dc5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8120158f40 0x7f81200ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:01.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.287+0000 7f8125dc5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f81200cf380 0x7f812006f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:01.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.287+0000 7f8125dc5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81200cd2b0 con 0x7f81200cb9d0 2023-11-24T06:17:01.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.287+0000 7f8125dc5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81200d02e0 con 0x7f8120158f40 2023-11-24T06:17:01.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.287+0000 7f8125dc5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812006f670 con 0x7f81200cf380 2023-11-24T06:17:01.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.287+0000 7f811f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8120158f40 0x7f81200ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:01.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.287+0000 7f811f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8120158f40 0x7f81200ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43044/0 (socket says 172.21.15.5:43044) 2023-11-24T06:17:01.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.287+0000 7f811f7fe640 1 -- 172.21.15.5:0/3735989308 learned_addr learned my addr 172.21.15.5:0/3735989308 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:01.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.287+0000 7f811ffff640 1 --2- 172.21.15.5:0/3735989308 >> v2:172.21.15.6:3300/0 conn(0x7f81200cf380 0x7f812006f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:01.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.287+0000 7f811f7fe640 1 -- 172.21.15.5:0/3735989308 >> v2:172.21.15.6:3300/0 conn(0x7f81200cf380 msgr2=0x7f812006f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:01.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.287+0000 7f811f7fe640 1 --2- 172.21.15.5:0/3735989308 >> v2:172.21.15.6:3300/0 conn(0x7f81200cf380 0x7f812006f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.287+0000 7f811f7fe640 1 -- 172.21.15.5:0/3735989308 >> v1:172.21.15.6:6789/0 conn(0x7f81200cb9d0 legacy=0x7f81200cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:01.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.287+0000 7f811f7fe640 1 -- 172.21.15.5:0/3735989308 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81200cbe30 con 0x7f8120158f40 2023-11-24T06:17:01.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.288+0000 7f811f7fe640 1 --2- 172.21.15.5:0/3735989308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8120158f40 0x7f81200ce6b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f812015c930 tx=0x7f811002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a70006a3498641f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:01.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.288+0000 7f811effd640 1 -- 172.21.15.5:0/3735989308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8110031870 con 0x7f8120158f40 2023-11-24T06:17:01.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.288+0000 7f811effd640 1 -- 172.21.15.5:0/3735989308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81100319d0 con 0x7f8120158f40 2023-11-24T06:17:01.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.288+0000 7f811effd640 1 -- 172.21.15.5:0/3735989308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8110031d60 con 0x7f8120158f40 2023-11-24T06:17:01.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.289+0000 7f8125dc5640 1 -- 172.21.15.5:0/3735989308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8120158f40 msgr2=0x7f81200ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:01.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.289+0000 7f8125dc5640 1 --2- 172.21.15.5:0/3735989308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8120158f40 0x7f81200ce6b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f812015c930 tx=0x7f811002b910 comp rx=0 tx=0).stop 2023-11-24T06:17:01.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.289+0000 7f8125dc5640 1 -- 172.21.15.5:0/3735989308 shutdown_connections 2023-11-24T06:17:01.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.289+0000 7f8125dc5640 1 --2- 172.21.15.5:0/3735989308 >> v2:172.21.15.6:3300/0 conn(0x7f81200cf380 0x7f812006f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.289+0000 7f8125dc5640 1 --2- 172.21.15.5:0/3735989308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8120158f40 0x7f81200ce6b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.289+0000 7f8125dc5640 1 -- 172.21.15.5:0/3735989308 >> 172.21.15.5:0/3735989308 conn(0x7f81200cdb20 msgr2=0x7f81200cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:01.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.289+0000 7f8125dc5640 1 -- 172.21.15.5:0/3735989308 shutdown_connections 2023-11-24T06:17:01.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.289+0000 7f8125dc5640 1 -- 172.21.15.5:0/3735989308 wait complete. 2023-11-24T06:17:01.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.292+0000 7f8125dc5640 1 Processor -- start 2023-11-24T06:17:01.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.292+0000 7f8125dc5640 1 -- start start 2023-11-24T06:17:01.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f8125dc5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81200cb9d0 0x7f81201480f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:01.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f8125dc5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81200cf380 0x7f81200c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:01.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f8125dc5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8120158f40 0x7f81200a8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:01.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f8125dc5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8120148630 con 0x7f81200cb9d0 2023-11-24T06:17:01.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f8125dc5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81200cab00 con 0x7f8120158f40 2023-11-24T06:17:01.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f8125dc5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81200ca540 con 0x7f81200cf380 2023-11-24T06:17:01.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f811f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81200cb9d0 0x7f81201480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:01.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f811f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81200cb9d0 0x7f81201480f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43060/0 (socket says 172.21.15.5:43060) 2023-11-24T06:17:01.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f811f7fe640 1 -- 172.21.15.5:0/3788423557 learned_addr learned my addr 172.21.15.5:0/3788423557 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:01.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f811ffff640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8120158f40 0x7f81200a8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:01.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f8117fff640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81200cf380 0x7f81200c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:01.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f811f7fe640 1 -- 172.21.15.5:0/3788423557 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81200cf380 msgr2=0x7f81200c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:01.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f811f7fe640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81200cf380 0x7f81200c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f811f7fe640 1 -- 172.21.15.5:0/3788423557 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8120158f40 msgr2=0x7f81200a8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:01.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f811f7fe640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8120158f40 0x7f81200a8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f811f7fe640 1 -- 172.21.15.5:0/3788423557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81100094d0 con 0x7f81200cb9d0 2023-11-24T06:17:01.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f8117fff640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81200cf380 0x7f81200c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:01.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f811f7fe640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81200cb9d0 0x7f81201480f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8110000c00 tx=0x7f81100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:01.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.293+0000 7f811cff9640 1 -- 172.21.15.5:0/3788423557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811003abe0 con 0x7f81200cb9d0 2023-11-24T06:17:01.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.294+0000 7f811cff9640 1 -- 172.21.15.5:0/3788423557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811003ad40 con 0x7f81200cb9d0 2023-11-24T06:17:01.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.294+0000 7f811cff9640 1 -- 172.21.15.5:0/3788423557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8110044610 con 0x7f81200cb9d0 2023-11-24T06:17:01.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.294+0000 7f8125dc5640 1 -- 172.21.15.5:0/3788423557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81200ca820 con 0x7f81200cb9d0 2023-11-24T06:17:01.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.294+0000 7f8125dc5640 1 -- 172.21.15.5:0/3788423557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8120148c90 con 0x7f81200cb9d0 2023-11-24T06:17:01.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.295+0000 7f8125dc5640 1 -- 172.21.15.5:0/3788423557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8120148e20 con 0x7f81200cb9d0 2023-11-24T06:17:01.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.296+0000 7f811cff9640 1 -- 172.21.15.5:0/3788423557 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f811004d450 con 0x7f81200cb9d0 2023-11-24T06:17:01.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.298+0000 7f811cff9640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f810006ae40 0x7f810006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:01.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.298+0000 7f811cff9640 1 -- 172.21.15.5:0/3788423557 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f811002d040 con 0x7f81200cb9d0 2023-11-24T06:17:01.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.298+0000 7f8117fff640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f810006ae40 0x7f810006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:01.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.298+0000 7f8117fff640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f810006ae40 0x7f810006d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f81201586b0 tx=0x7f8108009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:01.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.301+0000 7f811cff9640 1 -- 172.21.15.5:0/3788423557 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8110076c80 con 0x7f81200cb9d0 2023-11-24T06:17:01.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.495+0000 7f8125dc5640 1 -- 172.21.15.5:0/3788423557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f812006fb50 con 0x7f81200cb9d0 2023-11-24T06:17:01.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.496+0000 7f811cff9640 1 -- 172.21.15.5:0/3788423557 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f81100350a0 con 0x7f81200cb9d0 2023-11-24T06:17:01.499 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:17:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.499+0000 7f8125dc5640 1 -- 172.21.15.5:0/3788423557 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f810006ae40 msgr2=0x7f810006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.499+0000 7f8125dc5640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f810006ae40 0x7f810006d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f81201586b0 tx=0x7f8108009040 comp rx=0 tx=0).stop 2023-11-24T06:17:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.499+0000 7f8125dc5640 1 -- 172.21.15.5:0/3788423557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81200cb9d0 msgr2=0x7f81201480f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.499+0000 7f8125dc5640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81200cb9d0 0x7f81201480f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8110000c00 tx=0x7f81100046c0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.499+0000 7f8125dc5640 1 -- 172.21.15.5:0/3788423557 shutdown_connections 2023-11-24T06:17:01.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.499+0000 7f8125dc5640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f810006ae40 0x7f810006d300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.499+0000 7f8125dc5640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81200cf380 0x7f81200c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.499+0000 7f8125dc5640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8120158f40 0x7f81200a8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.499+0000 7f8125dc5640 1 --2- 172.21.15.5:0/3788423557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81200cb9d0 0x7f81201480f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.499+0000 7f8125dc5640 1 -- 172.21.15.5:0/3788423557 >> 172.21.15.5:0/3788423557 conn(0x7f81200cdb20 msgr2=0x7f812015aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.499+0000 7f8125dc5640 1 -- 172.21.15.5:0/3788423557 shutdown_connections 2023-11-24T06:17:01.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.499+0000 7f8125dc5640 1 -- 172.21.15.5:0/3788423557 wait complete. 2023-11-24T06:17:01.521 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:17:01.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.735+0000 7f789ddb3640 1 Processor -- start 2023-11-24T06:17:01.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.736+0000 7f789ddb3640 1 -- start start 2023-11-24T06:17:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.736+0000 7f789ddb3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f78981435d0 0x7f7898071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.736+0000 7f789ddb3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78981439b0 0x7f78980b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.737+0000 7f789ddb3640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7898070d40 con 0x7f78980b3f70 2023-11-24T06:17:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.737+0000 7f789ddb3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78980b4400 con 0x7f78981435d0 2023-11-24T06:17:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.737+0000 7f789ddb3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7898071020 con 0x7f78981439b0 2023-11-24T06:17:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.737+0000 7f789cdb1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f78981435d0 0x7f7898071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.737+0000 7f789cdb1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f78981435d0 0x7f7898071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43074/0 (socket says 172.21.15.5:43074) 2023-11-24T06:17:01.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.737+0000 7f789cdb1640 1 -- 172.21.15.5:0/1362841077 learned_addr learned my addr 172.21.15.5:0/1362841077 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:01.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.737+0000 7f789cdb1640 1 -- 172.21.15.5:0/1362841077 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78981439b0 msgr2=0x7f78980b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:01.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.737+0000 7f7897fff640 1 --2- 172.21.15.5:0/1362841077 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78981439b0 0x7f78980b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:01.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.737+0000 7f789cdb1640 1 --2- 172.21.15.5:0/1362841077 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78981439b0 0x7f78980b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.737+0000 7f789cdb1640 1 -- 172.21.15.5:0/1362841077 >> v1:172.21.15.6:6789/0 conn(0x7f78980b3f70 legacy=0x7f7898070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:01.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.737+0000 7f789cdb1640 1 -- 172.21.15.5:0/1362841077 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7898071300 con 0x7f78981435d0 2023-11-24T06:17:01.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.738+0000 7f789cdb1640 1 --2- 172.21.15.5:0/1362841077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78981435d0 0x7f7898071de0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f788c0098a0 tx=0x7f788c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29853224a929d9f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:01.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.739+0000 7f78977fe640 1 -- 172.21.15.5:0/1362841077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c031640 con 0x7f78981435d0 2023-11-24T06:17:01.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.739+0000 7f78977fe640 1 -- 172.21.15.5:0/1362841077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c0317a0 con 0x7f78981435d0 2023-11-24T06:17:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.739+0000 7f78977fe640 1 -- 172.21.15.5:0/1362841077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c031b30 con 0x7f78981435d0 2023-11-24T06:17:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.739+0000 7f789ddb3640 1 -- 172.21.15.5:0/1362841077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78981435d0 msgr2=0x7f7898071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.739+0000 7f789ddb3640 1 --2- 172.21.15.5:0/1362841077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78981435d0 0x7f7898071de0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f788c0098a0 tx=0x7f788c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.740+0000 7f789ddb3640 1 -- 172.21.15.5:0/1362841077 shutdown_connections 2023-11-24T06:17:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.740+0000 7f789ddb3640 1 --2- 172.21.15.5:0/1362841077 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78981439b0 0x7f78980b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.740+0000 7f789ddb3640 1 --2- 172.21.15.5:0/1362841077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78981435d0 0x7f7898071de0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.740+0000 7f789ddb3640 1 -- 172.21.15.5:0/1362841077 >> 172.21.15.5:0/1362841077 conn(0x7f78980cec10 msgr2=0x7f78980cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.740+0000 7f789ddb3640 1 -- 172.21.15.5:0/1362841077 shutdown_connections 2023-11-24T06:17:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.740+0000 7f789ddb3640 1 -- 172.21.15.5:0/1362841077 wait complete. 2023-11-24T06:17:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.741+0000 7f789ddb3640 1 Processor -- start 2023-11-24T06:17:01.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.742+0000 7f789ddb3640 1 -- start start 2023-11-24T06:17:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789ddb3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78980b3f70 0x7f78980cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789ddb3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78981435d0 0x7f7898139140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789ddb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78981439b0 0x7f7898139680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789ddb3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7898139bc0 con 0x7f78981439b0 2023-11-24T06:17:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789ddb3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7898157c60 con 0x7f78980b3f70 2023-11-24T06:17:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789ddb3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7898157f40 con 0x7f78981435d0 2023-11-24T06:17:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789d5b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78981439b0 0x7f7898139680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789d5b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78981439b0 0x7f7898139680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43080/0 (socket says 172.21.15.5:43080) 2023-11-24T06:17:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789d5b2640 1 -- 172.21.15.5:0/1360660312 learned_addr learned my addr 172.21.15.5:0/1360660312 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789cdb1640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78980b3f70 0x7f78980cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f7897fff640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78981435d0 0x7f7898139140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789d5b2640 1 -- 172.21.15.5:0/1360660312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78981435d0 msgr2=0x7f7898139140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789d5b2640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78981435d0 0x7f7898139140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789d5b2640 1 -- 172.21.15.5:0/1360660312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78980b3f70 msgr2=0x7f78980cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789d5b2640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78980b3f70 0x7f78980cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789d5b2640 1 -- 172.21.15.5:0/1360660312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c0094d0 con 0x7f78981439b0 2023-11-24T06:17:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.743+0000 7f789cdb1640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78980b3f70 0x7f78980cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.744+0000 7f789d5b2640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78981439b0 0x7f7898139680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f789000c780 tx=0x7f789000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.744+0000 7f78957fa640 1 -- 172.21.15.5:0/1360660312 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890003d80 con 0x7f78981439b0 2023-11-24T06:17:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.744+0000 7f78957fa640 1 -- 172.21.15.5:0/1360660312 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789000ce90 con 0x7f78981439b0 2023-11-24T06:17:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.744+0000 7f78957fa640 1 -- 172.21.15.5:0/1360660312 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78900052b0 con 0x7f78981439b0 2023-11-24T06:17:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.744+0000 7f789ddb3640 1 -- 172.21.15.5:0/1360660312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7898140a40 con 0x7f78981439b0 2023-11-24T06:17:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.744+0000 7f789ddb3640 1 -- 172.21.15.5:0/1360660312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7898141000 con 0x7f78981439b0 2023-11-24T06:17:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.746+0000 7f789ddb3640 1 -- 172.21.15.5:0/1360660312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7898141220 con 0x7f78981439b0 2023-11-24T06:17:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.746+0000 7f78957fa640 1 -- 172.21.15.5:0/1360660312 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7890005410 con 0x7f78981439b0 2023-11-24T06:17:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.747+0000 7f78957fa640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f787c06ae40 0x7f787c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.748+0000 7f78957fa640 1 -- 172.21.15.5:0/1360660312 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7890088ca0 con 0x7f78981439b0 2023-11-24T06:17:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.748+0000 7f789cdb1640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f787c06ae40 0x7f787c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.748+0000 7f789cdb1640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f787c06ae40 0x7f787c06d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f788c036000 tx=0x7f788c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:01.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.752+0000 7f78957fa640 1 -- 172.21.15.5:0/1360660312 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7890053b90 con 0x7f78981439b0 2023-11-24T06:17:01.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.953+0000 7f789ddb3640 1 -- 172.21.15.5:0/1360660312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f789804e840 con 0x7f78981439b0 2023-11-24T06:17:01.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.953+0000 7f78957fa640 1 -- 172.21.15.5:0/1360660312 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7890017070 con 0x7f78981439b0 2023-11-24T06:17:01.956 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:17:01.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.956+0000 7f789ddb3640 1 -- 172.21.15.5:0/1360660312 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f787c06ae40 msgr2=0x7f787c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:01.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.956+0000 7f789ddb3640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f787c06ae40 0x7f787c06d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f788c036000 tx=0x7f788c03c040 comp rx=0 tx=0).stop 2023-11-24T06:17:01.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.956+0000 7f789ddb3640 1 -- 172.21.15.5:0/1360660312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78981439b0 msgr2=0x7f7898139680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:01.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.956+0000 7f789ddb3640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78981439b0 0x7f7898139680 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f789000c780 tx=0x7f789000cc40 comp rx=0 tx=0).stop 2023-11-24T06:17:01.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.956+0000 7f789ddb3640 1 -- 172.21.15.5:0/1360660312 shutdown_connections 2023-11-24T06:17:01.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.956+0000 7f789ddb3640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f787c06ae40 0x7f787c06d300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.956+0000 7f789ddb3640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f78981435d0 0x7f7898139140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.956+0000 7f789ddb3640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78981439b0 0x7f7898139680 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.956+0000 7f789ddb3640 1 --2- 172.21.15.5:0/1360660312 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f78980b3f70 0x7f78980cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:01.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.956+0000 7f789ddb3640 1 -- 172.21.15.5:0/1360660312 >> 172.21.15.5:0/1360660312 conn(0x7f78980cec10 msgr2=0x7f7898070050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:01.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.956+0000 7f789ddb3640 1 -- 172.21.15.5:0/1360660312 shutdown_connections 2023-11-24T06:17:01.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:01.957+0000 7f789ddb3640 1 -- 172.21.15.5:0/1360660312 wait complete. 2023-11-24T06:17:01.978 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:17:02.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.203+0000 7f8a55183640 1 Processor -- start 2023-11-24T06:17:02.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.203+0000 7f8a55183640 1 -- start start 2023-11-24T06:17:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.205+0000 7f8a55183640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8a50157770 0x7f8a5006cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.205+0000 7f8a55183640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5006d080 0x7f8a5015d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.205+0000 7f8a55183640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8a5015ddb0 0x7f8a500ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.205+0000 7f8a55183640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5006d500 con 0x7f8a5015ddb0 2023-11-24T06:17:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.205+0000 7f8a55183640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a50161670 con 0x7f8a50157770 2023-11-24T06:17:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.205+0000 7f8a55183640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5015e610 con 0x7f8a5006d080 2023-11-24T06:17:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.205+0000 7f8a54982640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8a5015ddb0 0x7f8a500ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.205+0000 7f8a54982640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8a5015ddb0 0x7f8a500ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43092/0 (socket says 172.21.15.5:43092) 2023-11-24T06:17:02.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.205+0000 7f8a54982640 1 -- 172.21.15.5:0/2890280031 learned_addr learned my addr 172.21.15.5:0/2890280031 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:02.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.206+0000 7f8a4f7fe640 1 --2- 172.21.15.5:0/2890280031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5006d080 0x7f8a5015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.206+0000 7f8a4ffff640 1 --2- 172.21.15.5:0/2890280031 >> v2:172.21.15.6:3300/0 conn(0x7f8a50157770 0x7f8a5006cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.206+0000 7f8a54982640 1 -- 172.21.15.5:0/2890280031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5006d080 msgr2=0x7f8a5015d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:02.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.206+0000 7f8a54982640 1 --2- 172.21.15.5:0/2890280031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5006d080 0x7f8a5015d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.206+0000 7f8a54982640 1 -- 172.21.15.5:0/2890280031 >> v2:172.21.15.6:3300/0 conn(0x7f8a50157770 msgr2=0x7f8a5006cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:02.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.206+0000 7f8a54982640 1 --2- 172.21.15.5:0/2890280031 >> v2:172.21.15.6:3300/0 conn(0x7f8a50157770 0x7f8a5006cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.206+0000 7f8a54982640 1 -- 172.21.15.5:0/2890280031 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a50160d40 con 0x7f8a5015ddb0 2023-11-24T06:17:02.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.207+0000 7f8a54982640 1 --2- 172.21.15.5:0/2890280031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5015ddb0 0x7f8a500ac870 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8a500adb70 tx=0x7f8a4802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145913b276d45f9f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:02.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.207+0000 7f8a4effd640 1 -- 172.21.15.5:0/2890280031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4802ca70 con 0x7f8a5015ddb0 2023-11-24T06:17:02.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.207+0000 7f8a4effd640 1 -- 172.21.15.5:0/2890280031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4802cbd0 con 0x7f8a5015ddb0 2023-11-24T06:17:02.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.207+0000 7f8a4effd640 1 -- 172.21.15.5:0/2890280031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a480343f0 con 0x7f8a5015ddb0 2023-11-24T06:17:02.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.208+0000 7f8a55183640 1 -- 172.21.15.5:0/2890280031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5015ddb0 msgr2=0x7f8a500ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:02.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.208+0000 7f8a55183640 1 --2- 172.21.15.5:0/2890280031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5015ddb0 0x7f8a500ac870 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8a500adb70 tx=0x7f8a4802bed0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.208+0000 7f8a55183640 1 -- 172.21.15.5:0/2890280031 shutdown_connections 2023-11-24T06:17:02.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.208+0000 7f8a55183640 1 --2- 172.21.15.5:0/2890280031 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5006d080 0x7f8a5015d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.208+0000 7f8a55183640 1 --2- 172.21.15.5:0/2890280031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5015ddb0 0x7f8a500ac870 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.208+0000 7f8a55183640 1 --2- 172.21.15.5:0/2890280031 >> v2:172.21.15.6:3300/0 conn(0x7f8a50157770 0x7f8a5006cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.208+0000 7f8a55183640 1 -- 172.21.15.5:0/2890280031 >> 172.21.15.5:0/2890280031 conn(0x7f8a5006a860 msgr2=0x7f8a5006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:02.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.208+0000 7f8a55183640 1 -- 172.21.15.5:0/2890280031 shutdown_connections 2023-11-24T06:17:02.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.208+0000 7f8a55183640 1 -- 172.21.15.5:0/2890280031 wait complete. 2023-11-24T06:17:02.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.210+0000 7f8a55183640 1 Processor -- start 2023-11-24T06:17:02.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.210+0000 7f8a55183640 1 -- start start 2023-11-24T06:17:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.211+0000 7f8a55183640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5006d080 0x7f8a5015fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.211+0000 7f8a55183640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a50157770 0x7f8a500ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.211+0000 7f8a55183640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5015ddb0 0x7f8a50153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.211+0000 7f8a55183640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a501597d0 con 0x7f8a5015ddb0 2023-11-24T06:17:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.211+0000 7f8a55183640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a50158b50 con 0x7f8a50157770 2023-11-24T06:17:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.211+0000 7f8a55183640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a50158e30 con 0x7f8a5006d080 2023-11-24T06:17:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.211+0000 7f8a54982640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5015ddb0 0x7f8a50153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.211+0000 7f8a54982640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5015ddb0 0x7f8a50153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43106/0 (socket says 172.21.15.5:43106) 2023-11-24T06:17:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.211+0000 7f8a54982640 1 -- 172.21.15.5:0/1028969183 learned_addr learned my addr 172.21.15.5:0/1028969183 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:02.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.212+0000 7f8a4ffff640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5006d080 0x7f8a5015fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.212+0000 7f8a4f7fe640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a50157770 0x7f8a500ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.212+0000 7f8a54982640 1 -- 172.21.15.5:0/1028969183 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5006d080 msgr2=0x7f8a5015fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.212+0000 7f8a54982640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5006d080 0x7f8a5015fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.212+0000 7f8a54982640 1 -- 172.21.15.5:0/1028969183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a50157770 msgr2=0x7f8a500ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.212+0000 7f8a54982640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a50157770 0x7f8a500ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.212+0000 7f8a54982640 1 -- 172.21.15.5:0/1028969183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a48009b30 con 0x7f8a5015ddb0 2023-11-24T06:17:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.212+0000 7f8a4f7fe640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a50157770 0x7f8a500ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.212+0000 7f8a54982640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5015ddb0 0x7f8a50153e10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8a48036040 tx=0x7f8a48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.213+0000 7f8a4cff9640 1 -- 172.21.15.5:0/1028969183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48040670 con 0x7f8a5015ddb0 2023-11-24T06:17:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.213+0000 7f8a4cff9640 1 -- 172.21.15.5:0/1028969183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a480407d0 con 0x7f8a5015ddb0 2023-11-24T06:17:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.213+0000 7f8a4cff9640 1 -- 172.21.15.5:0/1028969183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48040b80 con 0x7f8a5015ddb0 2023-11-24T06:17:02.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.213+0000 7f8a55183640 1 -- 172.21.15.5:0/1028969183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a50159110 con 0x7f8a5015ddb0 2023-11-24T06:17:02.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.213+0000 7f8a55183640 1 -- 172.21.15.5:0/1028969183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5015ff50 con 0x7f8a5015ddb0 2023-11-24T06:17:02.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.214+0000 7f8a55183640 1 -- 172.21.15.5:0/1028969183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a50160110 con 0x7f8a5015ddb0 2023-11-24T06:17:02.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.215+0000 7f8a4cff9640 1 -- 172.21.15.5:0/1028969183 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a48040ce0 con 0x7f8a5015ddb0 2023-11-24T06:17:02.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.216+0000 7f8a4cff9640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8a3c06ae40 0x7f8a3c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.216+0000 7f8a4cff9640 1 -- 172.21.15.5:0/1028969183 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8a480ac680 con 0x7f8a5015ddb0 2023-11-24T06:17:02.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.216+0000 7f8a4ffff640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8a3c06ae40 0x7f8a3c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.217+0000 7f8a4ffff640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8a3c06ae40 0x7f8a3c06d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006fd0 tx=0x7f8a40008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:02.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.221+0000 7f8a4cff9640 1 -- 172.21.15.5:0/1028969183 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a48077730 con 0x7f8a5015ddb0 2023-11-24T06:17:02.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.453+0000 7f8a55183640 1 -- 172.21.15.5:0/1028969183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a5004e840 con 0x7f8a5015ddb0 2023-11-24T06:17:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.454+0000 7f8a4cff9640 1 -- 172.21.15.5:0/1028969183 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4895 (secure 0 0 0) 0x7f8a48009510 con 0x7f8a5015ddb0 2023-11-24T06:17:02.456 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:02.456 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4009460946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4009460946},{"type":"v1","addr":"172.21.15.5:6837","nonce":4009460946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2496926427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2496926427},{"type":"v1","addr":"172.21.15.5:6836","nonce":2496926427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3829754769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3829754769},{"type":"v1","addr":"172.21.15.6:6835","nonce":3829754769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:15:51.611541+0000","modified":"2023-11-24T06:15:54.664234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6833/2760317954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2760317954},{"type":"v1","addr":"172.21.15.6:6833","nonce":2760317954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:17:02.456 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2023-11-24T06:17:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.457+0000 7f8a55183640 1 -- 172.21.15.5:0/1028969183 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8a3c06ae40 msgr2=0x7f8a3c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.457+0000 7f8a55183640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8a3c06ae40 0x7f8a3c06d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006fd0 tx=0x7f8a40008040 comp rx=0 tx=0).stop 2023-11-24T06:17:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.457+0000 7f8a55183640 1 -- 172.21.15.5:0/1028969183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5015ddb0 msgr2=0x7f8a50153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.457+0000 7f8a55183640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5015ddb0 0x7f8a50153e10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8a48036040 tx=0x7f8a48004a40 comp rx=0 tx=0).stop 2023-11-24T06:17:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.457+0000 7f8a55183640 1 -- 172.21.15.5:0/1028969183 shutdown_connections 2023-11-24T06:17:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.457+0000 7f8a55183640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8a3c06ae40 0x7f8a3c06d300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.457+0000 7f8a55183640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5015ddb0 0x7f8a50153e10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.457+0000 7f8a55183640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8a50157770 0x7f8a500ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.457+0000 7f8a55183640 1 --2- 172.21.15.5:0/1028969183 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8a5006d080 0x7f8a5015fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.457+0000 7f8a55183640 1 -- 172.21.15.5:0/1028969183 >> 172.21.15.5:0/1028969183 conn(0x7f8a5006a860 msgr2=0x7f8a50155250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:02.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.457+0000 7f8a55183640 1 -- 172.21.15.5:0/1028969183 shutdown_connections 2023-11-24T06:17:02.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.457+0000 7f8a55183640 1 -- 172.21.15.5:0/1028969183 wait complete. 2023-11-24T06:17:02.479 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:17:02.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.703+0000 7f77750ee640 1 Processor -- start 2023-11-24T06:17:02.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.704+0000 7f77750ee640 1 -- start start 2023-11-24T06:17:02.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.705+0000 7f77750ee640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f777006d520 0x7f777006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.705+0000 7f77750ee640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777015c360 con 0x7f77700ad720 2023-11-24T06:17:02.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.705+0000 7f77750ee640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777015faf0 con 0x7f77700abff0 2023-11-24T06:17:02.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.705+0000 7f77750ee640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77700b41a0 con 0x7f777006d520 2023-11-24T06:17:02.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.706+0000 7f7767fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f77700ad720 0x7f777015ecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46318/0 (socket says 172.21.15.5:46318) 2023-11-24T06:17:02.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.706+0000 7f7767fff640 1 -- 172.21.15.5:0/1761093310 learned_addr learned my addr 172.21.15.5:0/1761093310 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:02.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.706+0000 7f776ed9d640 1 --2- 172.21.15.5:0/1761093310 >> v2:172.21.15.6:3300/0 conn(0x7f777006d520 0x7f777006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.706+0000 7f776e59c640 1 -- 172.21.15.5:0/1761093310 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277357215 0 0) 0x7f777015c360 con 0x7f77700ad720 2023-11-24T06:17:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.706+0000 7f776e59c640 1 -- 172.21.15.5:0/1761093310 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f775c003520 con 0x7f77700ad720 2023-11-24T06:17:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.707+0000 7f776e59c640 1 -- 172.21.15.5:0/1761093310 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3028521302 0 0) 0x7f775c003520 con 0x7f77700ad720 2023-11-24T06:17:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.707+0000 7f776e59c640 1 -- 172.21.15.5:0/1761093310 >> v2:172.21.15.6:3300/0 conn(0x7f777006d520 msgr2=0x7f777006d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.707+0000 7f776e59c640 1 --2- 172.21.15.5:0/1761093310 >> v2:172.21.15.6:3300/0 conn(0x7f777006d520 0x7f777006d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.707+0000 7f776e59c640 1 -- 172.21.15.5:0/1761093310 >> v1:172.21.15.6:6789/0 conn(0x7f77700abff0 legacy=0x7f777015f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.707+0000 7f776e59c640 1 -- 172.21.15.5:0/1761093310 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777006de40 con 0x7f77700ad720 2023-11-24T06:17:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.707+0000 7f776e59c640 1 -- 172.21.15.5:0/1761093310 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f7758001670 con 0x7f77700ad720 2023-11-24T06:17:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.707+0000 7f776e59c640 1 -- 172.21.15.5:0/1761093310 >> v1:172.21.15.5:6789/0 conn(0x7f77700ad720 legacy=0x7f777015ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.707+0000 7f776e59c640 1 --2- 172.21.15.5:0/1761093310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f775c003970 0x7f775c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.707+0000 7f776e59c640 1 --2- 172.21.15.5:0/1761093310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775c0047c0 0x7f775c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.707+0000 7f776e59c640 1 --2- 172.21.15.5:0/1761093310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f775c005180 0x7f775c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.707+0000 7f776e59c640 1 -- 172.21.15.5:0/1761093310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c00bff0 con 0x7f775c0047c0 2023-11-24T06:17:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.707+0000 7f776e59c640 1 -- 172.21.15.5:0/1761093310 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775c00c160 con 0x7f775c005180 2023-11-24T06:17:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.707+0000 7f776e59c640 1 -- 172.21.15.5:0/1761093310 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775c00c320 con 0x7f775c003970 2023-11-24T06:17:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.707+0000 7f7767fff640 1 --2- 172.21.15.5:0/1761093310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775c0047c0 0x7f775c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.708+0000 7f776f59e640 1 --2- 172.21.15.5:0/1761093310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f775c005180 0x7f775c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.708+0000 7f776ed9d640 1 --2- 172.21.15.5:0/1761093310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f775c003970 0x7f775c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.708+0000 7f7767fff640 1 -- 172.21.15.5:0/1761093310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f775c003970 msgr2=0x7f775c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.708+0000 7f7767fff640 1 --2- 172.21.15.5:0/1761093310 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f775c003970 0x7f775c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.708+0000 7f7767fff640 1 -- 172.21.15.5:0/1761093310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f775c005180 msgr2=0x7f775c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.708+0000 7f7767fff640 1 --2- 172.21.15.5:0/1761093310 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f775c005180 0x7f775c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.708+0000 7f7767fff640 1 -- 172.21.15.5:0/1761093310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c003520 con 0x7f775c0047c0 2023-11-24T06:17:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.708+0000 7f776f59e640 1 -- 172.21.15.5:0/1761093310 reap_dead start 2023-11-24T06:17:02.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.709+0000 7f7767fff640 1 --2- 172.21.15.5:0/1761093310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775c0047c0 0x7f775c004c40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7770070290 tx=0x7f775c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb2bb3f6b7f0257 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:02.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.710+0000 7f776e59c640 1 -- 172.21.15.5:0/1761093310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7758005ad0 con 0x7f775c0047c0 2023-11-24T06:17:02.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.710+0000 7f776e59c640 1 -- 172.21.15.5:0/1761093310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7758005c30 con 0x7f775c0047c0 2023-11-24T06:17:02.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.710+0000 7f776e59c640 1 -- 172.21.15.5:0/1761093310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775802ca90 con 0x7f775c0047c0 2023-11-24T06:17:02.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.710+0000 7f77750ee640 1 -- 172.21.15.5:0/1761093310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775c0047c0 msgr2=0x7f775c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:02.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.710+0000 7f77750ee640 1 --2- 172.21.15.5:0/1761093310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775c0047c0 0x7f775c004c40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7770070290 tx=0x7f775c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.710+0000 7f77750ee640 1 -- 172.21.15.5:0/1761093310 shutdown_connections 2023-11-24T06:17:02.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.710+0000 7f77750ee640 1 --2- 172.21.15.5:0/1761093310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775c0047c0 0x7f775c004c40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.710+0000 7f77750ee640 1 -- 172.21.15.5:0/1761093310 >> 172.21.15.5:0/1761093310 conn(0x7f77700b0d80 msgr2=0x7f77700b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:02.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.710+0000 7f77750ee640 1 -- 172.21.15.5:0/1761093310 shutdown_connections 2023-11-24T06:17:02.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.711+0000 7f77750ee640 1 -- 172.21.15.5:0/1761093310 wait complete. 2023-11-24T06:17:02.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.712+0000 7f77750ee640 1 Processor -- start 2023-11-24T06:17:02.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.713+0000 7f77750ee640 1 -- start start 2023-11-24T06:17:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.713+0000 7f77750ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775c0047c0 0x7f7770149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f77750ee640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f777015ab20 0x7f7770152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f77750ee640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77701533e0 0x7f77700cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f77750ee640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7770153870 con 0x7f775c0047c0 2023-11-24T06:17:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f77750ee640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77701497a0 con 0x7f77701533e0 2023-11-24T06:17:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f77750ee640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7770159ac0 con 0x7f777015ab20 2023-11-24T06:17:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f776ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775c0047c0 0x7f7770149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f776ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775c0047c0 0x7f7770149260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43124/0 (socket says 172.21.15.5:43124) 2023-11-24T06:17:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f776ed9d640 1 -- 172.21.15.5:0/3546035749 learned_addr learned my addr 172.21.15.5:0/3546035749 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f7767fff640 1 --2- 172.21.15.5:0/3546035749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f777015ab20 0x7f7770152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f776ed9d640 1 -- 172.21.15.5:0/3546035749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f777015ab20 msgr2=0x7f7770152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f776ed9d640 1 --2- 172.21.15.5:0/3546035749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f777015ab20 0x7f7770152ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f776ed9d640 1 -- 172.21.15.5:0/3546035749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77701533e0 msgr2=0x7f77700cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:02.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f776f59e640 1 --2- 172.21.15.5:0/3546035749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77701533e0 0x7f77700cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f776ed9d640 1 --2- 172.21.15.5:0/3546035749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77701533e0 0x7f77700cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.714+0000 7f776ed9d640 1 -- 172.21.15.5:0/3546035749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7770159d60 con 0x7f775c0047c0 2023-11-24T06:17:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.715+0000 7f776ed9d640 1 --2- 172.21.15.5:0/3546035749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775c0047c0 0x7f7770149260 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7760010790 tx=0x7f7760010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:02.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.715+0000 7f77677fe640 1 -- 172.21.15.5:0/3546035749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7760009630 con 0x7f775c0047c0 2023-11-24T06:17:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.715+0000 7f77677fe640 1 -- 172.21.15.5:0/3546035749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776000eb60 con 0x7f775c0047c0 2023-11-24T06:17:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.715+0000 7f77677fe640 1 -- 172.21.15.5:0/3546035749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7760010ea0 con 0x7f775c0047c0 2023-11-24T06:17:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.715+0000 7f77750ee640 1 -- 172.21.15.5:0/3546035749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777015a040 con 0x7f775c0047c0 2023-11-24T06:17:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.715+0000 7f77750ee640 1 -- 172.21.15.5:0/3546035749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777015a600 con 0x7f775c0047c0 2023-11-24T06:17:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.716+0000 7f77750ee640 1 -- 172.21.15.5:0/3546035749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777015b6b0 con 0x7f775c0047c0 2023-11-24T06:17:02.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.717+0000 7f77677fe640 1 -- 172.21.15.5:0/3546035749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f776001c400 con 0x7f775c0047c0 2023-11-24T06:17:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.718+0000 7f77677fe640 1 --2- 172.21.15.5:0/3546035749 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f775006ae40 0x7f775006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.718+0000 7f77677fe640 1 -- 172.21.15.5:0/3546035749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7760089b20 con 0x7f775c0047c0 2023-11-24T06:17:02.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.718+0000 7f7767fff640 1 --2- 172.21.15.5:0/3546035749 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f775006ae40 0x7f775006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:02.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.719+0000 7f7767fff640 1 --2- 172.21.15.5:0/3546035749 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f775006ae40 0x7f775006d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7758034000 tx=0x7f7758009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.722+0000 7f77677fe640 1 -- 172.21.15.5:0/3546035749 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7760054bd0 con 0x7f775c0047c0 2023-11-24T06:17:02.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.916+0000 7f77750ee640 1 -- 172.21.15.5:0/3546035749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7770074770 con 0x7f775c0047c0 2023-11-24T06:17:02.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.918+0000 7f77677fe640 1 -- 172.21.15.5:0/3546035749 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16658 (secure 0 0 0) 0x7f77600581f0 con 0x7f775c0047c0 2023-11-24T06:17:02.921 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:02.921 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:16:33.958286+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:17:02.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.921+0000 7f77750ee640 1 -- 172.21.15.5:0/3546035749 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f775006ae40 msgr2=0x7f775006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:02.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.921+0000 7f77750ee640 1 --2- 172.21.15.5:0/3546035749 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f775006ae40 0x7f775006d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7758034000 tx=0x7f7758009030 comp rx=0 tx=0).stop 2023-11-24T06:17:02.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.921+0000 7f77750ee640 1 -- 172.21.15.5:0/3546035749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775c0047c0 msgr2=0x7f7770149260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:02.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.921+0000 7f77750ee640 1 --2- 172.21.15.5:0/3546035749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775c0047c0 0x7f7770149260 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7760010790 tx=0x7f7760010c50 comp rx=0 tx=0).stop 2023-11-24T06:17:02.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.921+0000 7f77750ee640 1 -- 172.21.15.5:0/3546035749 shutdown_connections 2023-11-24T06:17:02.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.921+0000 7f77750ee640 1 --2- 172.21.15.5:0/3546035749 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f775006ae40 0x7f775006d300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.921+0000 7f77750ee640 1 --2- 172.21.15.5:0/3546035749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f777015ab20 0x7f7770152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.921+0000 7f77750ee640 1 --2- 172.21.15.5:0/3546035749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f77701533e0 0x7f77700cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.921+0000 7f77750ee640 1 --2- 172.21.15.5:0/3546035749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775c0047c0 0x7f7770149260 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.921+0000 7f77750ee640 1 -- 172.21.15.5:0/3546035749 >> 172.21.15.5:0/3546035749 conn(0x7f77700b0d80 msgr2=0x7f77700b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.921+0000 7f77750ee640 1 -- 172.21.15.5:0/3546035749 shutdown_connections 2023-11-24T06:17:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:02.921+0000 7f77750ee640 1 -- 172.21.15.5:0/3546035749 wait complete. 2023-11-24T06:17:02.944 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:17:02.944 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:17:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.172+0000 7f76f59ce640 1 Processor -- start 2023-11-24T06:17:03.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.172+0000 7f76f59ce640 1 -- start start 2023-11-24T06:17:03.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.173+0000 7f76f59ce640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f76f015d480 0x7f76f0161050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.173+0000 7f76f59ce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76f0161590 0x7f76f00ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.173+0000 7f76f59ce640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f00ce9d0 con 0x7f76f00ce580 2023-11-24T06:17:03.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.173+0000 7f76f59ce640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f0161b80 con 0x7f76f015d480 2023-11-24T06:17:03.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.174+0000 7f76f59ce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f0161e60 con 0x7f76f0161590 2023-11-24T06:17:03.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.174+0000 7f76eeffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f76f015d480 0x7f76f0161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:03.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.174+0000 7f76eeffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f76f015d480 0x7f76f0161050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53428/0 (socket says 172.21.15.5:53428) 2023-11-24T06:17:03.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.174+0000 7f76eeffd640 1 -- 172.21.15.5:0/952211448 learned_addr learned my addr 172.21.15.5:0/952211448 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:03.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.174+0000 7f76eeffd640 1 -- 172.21.15.5:0/952211448 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76f0161590 msgr2=0x7f76f00ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:03.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.174+0000 7f76eeffd640 1 --2- 172.21.15.5:0/952211448 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76f0161590 0x7f76f00ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.174+0000 7f76eeffd640 1 -- 172.21.15.5:0/952211448 >> v1:172.21.15.6:6789/0 conn(0x7f76f00ce580 legacy=0x7f76f00b4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:03.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.174+0000 7f76eeffd640 1 -- 172.21.15.5:0/952211448 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f00d1360 con 0x7f76f015d480 2023-11-24T06:17:03.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.175+0000 7f76eeffd640 1 --2- 172.21.15.5:0/952211448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f015d480 0x7f76f0161050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009e50 tx=0x7f76e002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e0f2cab9477c95 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:03.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.176+0000 7f76edffb640 1 -- 172.21.15.5:0/952211448 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e002ca70 con 0x7f76f015d480 2023-11-24T06:17:03.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.176+0000 7f76edffb640 1 -- 172.21.15.5:0/952211448 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e002cbd0 con 0x7f76f015d480 2023-11-24T06:17:03.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.176+0000 7f76edffb640 1 -- 172.21.15.5:0/952211448 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e00343f0 con 0x7f76f015d480 2023-11-24T06:17:03.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.176+0000 7f76f59ce640 1 -- 172.21.15.5:0/952211448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f015d480 msgr2=0x7f76f0161050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:03.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.176+0000 7f76f59ce640 1 --2- 172.21.15.5:0/952211448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f015d480 0x7f76f0161050 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009e50 tx=0x7f76e002bed0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.176+0000 7f76f59ce640 1 -- 172.21.15.5:0/952211448 shutdown_connections 2023-11-24T06:17:03.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.176+0000 7f76f59ce640 1 --2- 172.21.15.5:0/952211448 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76f0161590 0x7f76f00ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.176+0000 7f76f59ce640 1 --2- 172.21.15.5:0/952211448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f015d480 0x7f76f0161050 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.176+0000 7f76f59ce640 1 -- 172.21.15.5:0/952211448 >> 172.21.15.5:0/952211448 conn(0x7f76f006b420 msgr2=0x7f76f006b830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:03.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.176+0000 7f76f59ce640 1 -- 172.21.15.5:0/952211448 shutdown_connections 2023-11-24T06:17:03.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.176+0000 7f76f59ce640 1 -- 172.21.15.5:0/952211448 wait complete. 2023-11-24T06:17:03.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.178+0000 7f76f59ce640 1 Processor -- start 2023-11-24T06:17:03.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.178+0000 7f76f59ce640 1 -- start start 2023-11-24T06:17:03.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.179+0000 7f76f59ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f00ce580 0x7f76f00ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.179+0000 7f76f59ce640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76f015d480 0x7f76f0160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.179+0000 7f76f59ce640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76f0161590 0x7f76f00cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.179+0000 7f76f59ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f01606b0 con 0x7f76f00ce580 2023-11-24T06:17:03.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.179+0000 7f76f59ce640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00ca920 con 0x7f76f0161590 2023-11-24T06:17:03.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.179+0000 7f76f59ce640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00cac00 con 0x7f76f015d480 2023-11-24T06:17:03.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.180+0000 7f76eeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f00ce580 0x7f76f00ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:03.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.180+0000 7f76eeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f00ce580 0x7f76f00ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53430/0 (socket says 172.21.15.5:53430) 2023-11-24T06:17:03.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.180+0000 7f76eeffd640 1 -- 172.21.15.5:0/2115056044 learned_addr learned my addr 172.21.15.5:0/2115056044 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:03.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.180+0000 7f76ef7fe640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76f0161590 0x7f76f00cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:03.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.180+0000 7f76eeffd640 1 -- 172.21.15.5:0/2115056044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76f015d480 msgr2=0x7f76f0160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:03.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.180+0000 7f76ee7fc640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76f015d480 0x7f76f0160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:03.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.180+0000 7f76eeffd640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76f015d480 0x7f76f0160170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.180+0000 7f76eeffd640 1 -- 172.21.15.5:0/2115056044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76f0161590 msgr2=0x7f76f00cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:03.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.180+0000 7f76eeffd640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76f0161590 0x7f76f00cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.180+0000 7f76eeffd640 1 -- 172.21.15.5:0/2115056044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e0009b30 con 0x7f76f00ce580 2023-11-24T06:17:03.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.180+0000 7f76ef7fe640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76f0161590 0x7f76f00cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:03.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.181+0000 7f76eeffd640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f00ce580 0x7f76f00ca340 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009e50 tx=0x7f76e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:03.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.181+0000 7f76e7fff640 1 -- 172.21.15.5:0/2115056044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e003e450 con 0x7f76f00ce580 2023-11-24T06:17:03.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.181+0000 7f76e7fff640 1 -- 172.21.15.5:0/2115056044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e003e5f0 con 0x7f76f00ce580 2023-11-24T06:17:03.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.181+0000 7f76f59ce640 1 -- 172.21.15.5:0/2115056044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f00caee0 con 0x7f76f00ce580 2023-11-24T06:17:03.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.181+0000 7f76e7fff640 1 -- 172.21.15.5:0/2115056044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e003e9a0 con 0x7f76f00ce580 2023-11-24T06:17:03.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.181+0000 7f76f59ce640 1 -- 172.21.15.5:0/2115056044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f00cb4a0 con 0x7f76f00ce580 2023-11-24T06:17:03.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.182+0000 7f76f59ce640 1 -- 172.21.15.5:0/2115056044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f00cb690 con 0x7f76f00ce580 2023-11-24T06:17:03.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.183+0000 7f76e7fff640 1 -- 172.21.15.5:0/2115056044 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f76e003eb00 con 0x7f76f00ce580 2023-11-24T06:17:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.185+0000 7f76e7fff640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f76d006ae40 0x7f76d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.185+0000 7f76e7fff640 1 -- 172.21.15.5:0/2115056044 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f76e00ac4a0 con 0x7f76f00ce580 2023-11-24T06:17:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.185+0000 7f76ee7fc640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f76d006ae40 0x7f76d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.186+0000 7f76ee7fc640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f76d006ae40 0x7f76d006d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f76f015eed0 tx=0x7f76d8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:03.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.190+0000 7f76e7fff640 1 -- 172.21.15.5:0/2115056044 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f76e0077420 con 0x7f76f00ce580 2023-11-24T06:17:03.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.424+0000 7f76f59ce640 1 -- 172.21.15.5:0/2115056044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76f00ce9d0 con 0x7f76f00ce580 2023-11-24T06:17:03.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.425+0000 7f76e7fff640 1 -- 172.21.15.5:0/2115056044 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4895 (secure 0 0 0) 0x7f76f00ce9d0 con 0x7f76f00ce580 2023-11-24T06:17:03.427 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:03.427 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4009460946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4009460946},{"type":"v1","addr":"172.21.15.5:6837","nonce":4009460946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2496926427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2496926427},{"type":"v1","addr":"172.21.15.5:6836","nonce":2496926427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3829754769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3829754769},{"type":"v1","addr":"172.21.15.6:6835","nonce":3829754769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:15:51.611541+0000","modified":"2023-11-24T06:15:54.664234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6833/2760317954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2760317954},{"type":"v1","addr":"172.21.15.6:6833","nonce":2760317954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:17:03.428 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2023-11-24T06:17:03.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.428+0000 7f76f59ce640 1 -- 172.21.15.5:0/2115056044 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f76d006ae40 msgr2=0x7f76d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:03.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.428+0000 7f76f59ce640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f76d006ae40 0x7f76d006d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f76f015eed0 tx=0x7f76d8008040 comp rx=0 tx=0).stop 2023-11-24T06:17:03.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.428+0000 7f76f59ce640 1 -- 172.21.15.5:0/2115056044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f00ce580 msgr2=0x7f76f00ca340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:03.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.428+0000 7f76f59ce640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f00ce580 0x7f76f00ca340 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f76e0009e50 tx=0x7f76e0004a40 comp rx=0 tx=0).stop 2023-11-24T06:17:03.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.428+0000 7f76f59ce640 1 -- 172.21.15.5:0/2115056044 shutdown_connections 2023-11-24T06:17:03.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.428+0000 7f76f59ce640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f76d006ae40 0x7f76d006d300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.428+0000 7f76f59ce640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76f015d480 0x7f76f0160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.428+0000 7f76f59ce640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76f0161590 0x7f76f00cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.428+0000 7f76f59ce640 1 --2- 172.21.15.5:0/2115056044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f00ce580 0x7f76f00ca340 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.428+0000 7f76f59ce640 1 -- 172.21.15.5:0/2115056044 >> 172.21.15.5:0/2115056044 conn(0x7f76f006b420 msgr2=0x7f76f00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:03.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.428+0000 7f76f59ce640 1 -- 172.21.15.5:0/2115056044 shutdown_connections 2023-11-24T06:17:03.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.428+0000 7f76f59ce640 1 -- 172.21.15.5:0/2115056044 wait complete. 2023-11-24T06:17:03.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:17:03.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.678+0000 7ff9c7368640 1 Processor -- start 2023-11-24T06:17:03.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.678+0000 7ff9c7368640 1 -- start start 2023-11-24T06:17:03.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.679+0000 7ff9c7368640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff9c015a100 0x7ff9c00b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.679+0000 7ff9c7368640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9c00ae7e0 0x7ff9c00b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.679+0000 7ff9c7368640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c013da30 con 0x7ff9c00abff0 2023-11-24T06:17:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.679+0000 7ff9c7368640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9c00ac840 con 0x7ff9c015a100 2023-11-24T06:17:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.679+0000 7ff9c7368640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c00ac560 con 0x7ff9c00ae7e0 2023-11-24T06:17:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.680+0000 7ff9c4903640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff9c00abff0 0x7ff9c006d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59088/0 (socket says 172.21.15.5:59088) 2023-11-24T06:17:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.680+0000 7ff9c4903640 1 -- 172.21.15.5:0/3388235083 learned_addr learned my addr 172.21.15.5:0/3388235083 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:03.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.680+0000 7ff9c5104640 1 --2- 172.21.15.5:0/3388235083 >> v2:172.21.15.6:3300/0 conn(0x7ff9c015a100 0x7ff9c00b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:03.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.680+0000 7ff9c5905640 1 --2- 172.21.15.5:0/3388235083 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9c00ae7e0 0x7ff9c00b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:03.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.680+0000 7ff9b7fff640 1 -- 172.21.15.5:0/3388235083 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261478473 0 0) 0x7ff9c013da30 con 0x7ff9c00abff0 2023-11-24T06:17:03.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.680+0000 7ff9b7fff640 1 -- 172.21.15.5:0/3388235083 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9ac003520 con 0x7ff9c00abff0 2023-11-24T06:17:03.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.680+0000 7ff9b7fff640 1 -- 172.21.15.5:0/3388235083 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4246890700 0 0) 0x7ff9ac003520 con 0x7ff9c00abff0 2023-11-24T06:17:03.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.680+0000 7ff9b7fff640 1 -- 172.21.15.5:0/3388235083 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9c00ae7e0 msgr2=0x7ff9c00b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:03.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.680+0000 7ff9b7fff640 1 --2- 172.21.15.5:0/3388235083 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9c00ae7e0 0x7ff9c00b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.680+0000 7ff9b7fff640 1 -- 172.21.15.5:0/3388235083 >> v2:172.21.15.6:3300/0 conn(0x7ff9c015a100 msgr2=0x7ff9c00b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:03.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.680+0000 7ff9b7fff640 1 --2- 172.21.15.5:0/3388235083 >> v2:172.21.15.6:3300/0 conn(0x7ff9c015a100 0x7ff9c00b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.680+0000 7ff9b7fff640 1 -- 172.21.15.5:0/3388235083 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c0070650 con 0x7ff9c00abff0 2023-11-24T06:17:03.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9b7fff640 1 -- 172.21.15.5:0/3388235083 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7ff9b0001670 con 0x7ff9c00abff0 2023-11-24T06:17:03.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9b7fff640 1 -- 172.21.15.5:0/3388235083 >> v1:172.21.15.5:6789/0 conn(0x7ff9c00abff0 legacy=0x7ff9c006d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:03.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9b7fff640 1 --2- 172.21.15.5:0/3388235083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9ac003970 0x7ff9ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9b7fff640 1 --2- 172.21.15.5:0/3388235083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9ac0047c0 0x7ff9ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9b7fff640 1 --2- 172.21.15.5:0/3388235083 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9c00ae7e0 0x7ff9ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9b7fff640 1 -- 172.21.15.5:0/3388235083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c00ac840 con 0x7ff9ac0047c0 2023-11-24T06:17:03.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9b7fff640 1 -- 172.21.15.5:0/3388235083 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c00ac560 con 0x7ff9ac003970 2023-11-24T06:17:03.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9b7fff640 1 -- 172.21.15.5:0/3388235083 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac00b980 con 0x7ff9c00ae7e0 2023-11-24T06:17:03.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9c4903640 1 --2- 172.21.15.5:0/3388235083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9ac0047c0 0x7ff9ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:03.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9c5104640 1 --2- 172.21.15.5:0/3388235083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9ac003970 0x7ff9ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:03.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9c5905640 1 --2- 172.21.15.5:0/3388235083 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9c00ae7e0 0x7ff9ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:03.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9c4903640 1 -- 172.21.15.5:0/3388235083 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9c00ae7e0 msgr2=0x7ff9ac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:03.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9c4903640 1 --2- 172.21.15.5:0/3388235083 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9c00ae7e0 0x7ff9ac005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9c4903640 1 -- 172.21.15.5:0/3388235083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9ac003970 msgr2=0x7ff9ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:03.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9c4903640 1 --2- 172.21.15.5:0/3388235083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9ac003970 0x7ff9ac003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9c4903640 1 -- 172.21.15.5:0/3388235083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac003520 con 0x7ff9ac0047c0 2023-11-24T06:17:03.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.681+0000 7ff9c5104640 1 --2- 172.21.15.5:0/3388235083 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9ac003970 0x7ff9ac003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:03.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.683+0000 7ff9c4903640 1 --2- 172.21.15.5:0/3388235083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9ac0047c0 0x7ff9ac004c40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000c110 tx=0x7ff9ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f573d2ca385c5bf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:03.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.683+0000 7ff9b7fff640 1 -- 172.21.15.5:0/3388235083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0005ad0 con 0x7ff9ac0047c0 2023-11-24T06:17:03.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.683+0000 7ff9b7fff640 1 -- 172.21.15.5:0/3388235083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b0005c30 con 0x7ff9ac0047c0 2023-11-24T06:17:03.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.683+0000 7ff9b7fff640 1 -- 172.21.15.5:0/3388235083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b002c810 con 0x7ff9ac0047c0 2023-11-24T06:17:03.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.683+0000 7ff9c7368640 1 -- 172.21.15.5:0/3388235083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9ac0047c0 msgr2=0x7ff9ac004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:03.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.683+0000 7ff9c7368640 1 --2- 172.21.15.5:0/3388235083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9ac0047c0 0x7ff9ac004c40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000c110 tx=0x7ff9ac0057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.683+0000 7ff9c5905640 1 -- 172.21.15.5:0/3388235083 reap_dead start 2023-11-24T06:17:03.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.684+0000 7ff9c7368640 1 -- 172.21.15.5:0/3388235083 shutdown_connections 2023-11-24T06:17:03.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.684+0000 7ff9c7368640 1 -- 172.21.15.5:0/3388235083 >> 172.21.15.5:0/3388235083 conn(0x7ff9c00b0d80 msgr2=0x7ff9c00b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:03.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.684+0000 7ff9c7368640 1 -- 172.21.15.5:0/3388235083 shutdown_connections 2023-11-24T06:17:03.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.684+0000 7ff9c7368640 1 -- 172.21.15.5:0/3388235083 wait complete. 2023-11-24T06:17:03.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.686+0000 7ff9c7368640 1 Processor -- start 2023-11-24T06:17:03.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.686+0000 7ff9c7368640 1 -- start start 2023-11-24T06:17:03.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c7368640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9ac0047c0 0x7ff9c00ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c7368640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9c00ce4e0 0x7ff9c00cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c7368640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c00ce960 0x7ff9c0152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c7368640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c00cba90 con 0x7ff9c00ce960 2023-11-24T06:17:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c7368640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c00cbd70 con 0x7ff9ac0047c0 2023-11-24T06:17:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c7368640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c00cc050 con 0x7ff9c00ce4e0 2023-11-24T06:17:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c5905640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c00ce960 0x7ff9c0152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c5905640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c00ce960 0x7ff9c0152bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53440/0 (socket says 172.21.15.5:53440) 2023-11-24T06:17:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c5905640 1 -- 172.21.15.5:0/1160304442 learned_addr learned my addr 172.21.15.5:0/1160304442 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c5104640 1 --2- 172.21.15.5:0/1160304442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9ac0047c0 0x7ff9c00ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c4903640 1 --2- 172.21.15.5:0/1160304442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9c00ce4e0 0x7ff9c00cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c5905640 1 -- 172.21.15.5:0/1160304442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9c00ce4e0 msgr2=0x7ff9c00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c5905640 1 --2- 172.21.15.5:0/1160304442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9c00ce4e0 0x7ff9c00cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c5905640 1 -- 172.21.15.5:0/1160304442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9ac0047c0 msgr2=0x7ff9c00ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c5905640 1 --2- 172.21.15.5:0/1160304442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9ac0047c0 0x7ff9c00ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.687+0000 7ff9c5905640 1 -- 172.21.15.5:0/1160304442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c00cc330 con 0x7ff9c00ce960 2023-11-24T06:17:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.688+0000 7ff9c5905640 1 --2- 172.21.15.5:0/1160304442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c00ce960 0x7ff9c0152bf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff9c015dbb0 tx=0x7ff9b800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.688+0000 7ff9b5ffb640 1 -- 172.21.15.5:0/1160304442 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8017400 con 0x7ff9c00ce960 2023-11-24T06:17:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.688+0000 7ff9b5ffb640 1 -- 172.21.15.5:0/1160304442 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b8011040 con 0x7ff9c00ce960 2023-11-24T06:17:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.688+0000 7ff9b5ffb640 1 -- 172.21.15.5:0/1160304442 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b80177c0 con 0x7ff9c00ce960 2023-11-24T06:17:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.688+0000 7ff9c7368640 1 -- 172.21.15.5:0/1160304442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c00cc610 con 0x7ff9c00ce960 2023-11-24T06:17:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.688+0000 7ff9c7368640 1 -- 172.21.15.5:0/1160304442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c00cdd80 con 0x7ff9c00ce960 2023-11-24T06:17:03.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.689+0000 7ff9c7368640 1 -- 172.21.15.5:0/1160304442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c0067960 con 0x7ff9c00ce960 2023-11-24T06:17:03.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.690+0000 7ff9b5ffb640 1 -- 172.21.15.5:0/1160304442 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff9b8017920 con 0x7ff9c00ce960 2023-11-24T06:17:03.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.691+0000 7ff9b5ffb640 1 --2- 172.21.15.5:0/1160304442 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff9a406ae40 0x7ff9a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:03.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.691+0000 7ff9b5ffb640 1 -- 172.21.15.5:0/1160304442 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff9b8089a80 con 0x7ff9c00ce960 2023-11-24T06:17:03.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.692+0000 7ff9c5104640 1 --2- 172.21.15.5:0/1160304442 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff9a406ae40 0x7ff9a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:03.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.692+0000 7ff9c5104640 1 --2- 172.21.15.5:0/1160304442 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff9a406ae40 0x7ff9a406d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009b10 tx=0x7ff9bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:03.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.696+0000 7ff9b5ffb640 1 -- 172.21.15.5:0/1160304442 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff9b8054a90 con 0x7ff9c00ce960 2023-11-24T06:17:03.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.931+0000 7ff9c7368640 1 -- 172.21.15.5:0/1160304442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9c00cedd0 con 0x7ff9c00ce960 2023-11-24T06:17:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.932+0000 7ff9b5ffb640 1 -- 172.21.15.5:0/1160304442 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4895 (secure 0 0 0) 0x7ff9b801c090 con 0x7ff9c00ce960 2023-11-24T06:17:03.934 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:03.935 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4223,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4009460946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":4009460946},{"type":"v1","addr":"172.21.15.5:6837","nonce":4009460946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2496926427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2496926427},{"type":"v1","addr":"172.21.15.5:6836","nonce":2496926427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3829754769","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3829754769},{"type":"v1","addr":"172.21.15.6:6835","nonce":3829754769}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:15:51.611541+0000","modified":"2023-11-24T06:15:54.664234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.6:6833/2760317954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2760317954},{"type":"v1","addr":"172.21.15.6:6833","nonce":2760317954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T06:17:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2023-11-24T06:17:03.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.935+0000 7ff9c7368640 1 -- 172.21.15.5:0/1160304442 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff9a406ae40 msgr2=0x7ff9a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.935+0000 7ff9c7368640 1 --2- 172.21.15.5:0/1160304442 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff9a406ae40 0x7ff9a406d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc009b10 tx=0x7ff9bc008040 comp rx=0 tx=0).stop 2023-11-24T06:17:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.935+0000 7ff9c7368640 1 -- 172.21.15.5:0/1160304442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c00ce960 msgr2=0x7ff9c0152bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.935+0000 7ff9c7368640 1 --2- 172.21.15.5:0/1160304442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c00ce960 0x7ff9c0152bf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff9c015dbb0 tx=0x7ff9b800f4a0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.935+0000 7ff9c7368640 1 -- 172.21.15.5:0/1160304442 shutdown_connections 2023-11-24T06:17:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.935+0000 7ff9c7368640 1 --2- 172.21.15.5:0/1160304442 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff9a406ae40 0x7ff9a406d300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.935+0000 7ff9c7368640 1 --2- 172.21.15.5:0/1160304442 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9c00ce4e0 0x7ff9c00cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.935+0000 7ff9c7368640 1 --2- 172.21.15.5:0/1160304442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c00ce960 0x7ff9c0152bf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.935+0000 7ff9c7368640 1 --2- 172.21.15.5:0/1160304442 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff9ac0047c0 0x7ff9c00ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.935+0000 7ff9c7368640 1 -- 172.21.15.5:0/1160304442 >> 172.21.15.5:0/1160304442 conn(0x7ff9c00b0d80 msgr2=0x7ff9c006df30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.936+0000 7ff9c7368640 1 -- 172.21.15.5:0/1160304442 shutdown_connections 2023-11-24T06:17:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:03.936+0000 7ff9c7368640 1 -- 172.21.15.5:0/1160304442 wait complete. 2023-11-24T06:17:03.956 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:17:04.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.166+0000 7f7a3b451640 1 Processor -- start 2023-11-24T06:17:04.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.166+0000 7f7a3b451640 1 -- start start 2023-11-24T06:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.168+0000 7f7a3b451640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7a34070780 0x7f7a3406d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.168+0000 7f7a3b451640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7a34070cd0 0x7f7a3406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.168+0000 7f7a3b451640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a34071cd0 con 0x7f7a34160970 2023-11-24T06:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.168+0000 7f7a3b451640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a340b2120 con 0x7f7a34070cd0 2023-11-24T06:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.168+0000 7f7a3b451640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a340b2400 con 0x7f7a34070780 2023-11-24T06:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.169+0000 7f7a399ee640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7a34070cd0 0x7f7a3406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.169+0000 7f7a399ee640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7a34070cd0 0x7f7a3406da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53454/0 (socket says 172.21.15.5:53454) 2023-11-24T06:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.169+0000 7f7a399ee640 1 -- 172.21.15.5:0/1439475135 learned_addr learned my addr 172.21.15.5:0/1439475135 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.169+0000 7f7a389ec640 1 --2- 172.21.15.5:0/1439475135 >> v2:172.21.15.6:3300/0 conn(0x7f7a34070780 0x7f7a3406d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.169+0000 7f7a399ee640 1 -- 172.21.15.5:0/1439475135 >> v2:172.21.15.6:3300/0 conn(0x7f7a34070780 msgr2=0x7f7a3406d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.169+0000 7f7a399ee640 1 --2- 172.21.15.5:0/1439475135 >> v2:172.21.15.6:3300/0 conn(0x7f7a34070780 0x7f7a3406d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.169+0000 7f7a399ee640 1 -- 172.21.15.5:0/1439475135 >> v1:172.21.15.6:6789/0 conn(0x7f7a34160970 legacy=0x7f7a340b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.169+0000 7f7a399ee640 1 -- 172.21.15.5:0/1439475135 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a340b26e0 con 0x7f7a34070cd0 2023-11-24T06:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.170+0000 7f7a399ee640 1 --2- 172.21.15.5:0/1439475135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a34070cd0 0x7f7a3406da60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7a3406b3f0 tx=0x7f7a2c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee9eaadc2b75f42 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.171+0000 7f7a2bfff640 1 -- 172.21.15.5:0/1439475135 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c02c9f0 con 0x7f7a34070cd0 2023-11-24T06:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.171+0000 7f7a2bfff640 1 -- 172.21.15.5:0/1439475135 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c02cb50 con 0x7f7a34070cd0 2023-11-24T06:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.171+0000 7f7a2bfff640 1 -- 172.21.15.5:0/1439475135 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0343f0 con 0x7f7a34070cd0 2023-11-24T06:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.171+0000 7f7a3b451640 1 -- 172.21.15.5:0/1439475135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a34070cd0 msgr2=0x7f7a3406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.171+0000 7f7a3b451640 1 --2- 172.21.15.5:0/1439475135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a34070cd0 0x7f7a3406da60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7a3406b3f0 tx=0x7f7a2c02be50 comp rx=0 tx=0).stop 2023-11-24T06:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.171+0000 7f7a3b451640 1 -- 172.21.15.5:0/1439475135 shutdown_connections 2023-11-24T06:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.171+0000 7f7a3b451640 1 --2- 172.21.15.5:0/1439475135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a34070cd0 0x7f7a3406da60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.171+0000 7f7a3b451640 1 --2- 172.21.15.5:0/1439475135 >> v2:172.21.15.6:3300/0 conn(0x7f7a34070780 0x7f7a3406d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.171+0000 7f7a3b451640 1 -- 172.21.15.5:0/1439475135 >> 172.21.15.5:0/1439475135 conn(0x7f7a340b0d80 msgr2=0x7f7a340b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.171+0000 7f7a3b451640 1 -- 172.21.15.5:0/1439475135 shutdown_connections 2023-11-24T06:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.171+0000 7f7a3b451640 1 -- 172.21.15.5:0/1439475135 wait complete. 2023-11-24T06:17:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.173+0000 7f7a3b451640 1 Processor -- start 2023-11-24T06:17:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.173+0000 7f7a3b451640 1 -- start start 2023-11-24T06:17:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.174+0000 7f7a3b451640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a34070780 0x7f7a3415c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.174+0000 7f7a3b451640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a34070cd0 0x7f7a34157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.174+0000 7f7a3b451640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a34160970 0x7f7a3409c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.174+0000 7f7a3b451640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a341594a0 con 0x7f7a34070cd0 2023-11-24T06:17:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.174+0000 7f7a3b451640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a34159780 con 0x7f7a34070780 2023-11-24T06:17:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.174+0000 7f7a3b451640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a34159a60 con 0x7f7a34160970 2023-11-24T06:17:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.175+0000 7f7a389ec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a34070cd0 0x7f7a34157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.175+0000 7f7a389ec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a34070cd0 0x7f7a34157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53462/0 (socket says 172.21.15.5:53462) 2023-11-24T06:17:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.175+0000 7f7a389ec640 1 -- 172.21.15.5:0/1850605646 learned_addr learned my addr 172.21.15.5:0/1850605646 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.175+0000 7f7a389ec640 1 -- 172.21.15.5:0/1850605646 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a34160970 msgr2=0x7f7a3409c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.175+0000 7f7a389ec640 1 --2- 172.21.15.5:0/1850605646 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a34160970 0x7f7a3409c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.175+0000 7f7a389ec640 1 -- 172.21.15.5:0/1850605646 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a34070780 msgr2=0x7f7a3415c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.175+0000 7f7a389ec640 1 --2- 172.21.15.5:0/1850605646 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a34070780 0x7f7a3415c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.175+0000 7f7a389ec640 1 -- 172.21.15.5:0/1850605646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c009b30 con 0x7f7a34070cd0 2023-11-24T06:17:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.175+0000 7f7a389ec640 1 --2- 172.21.15.5:0/1850605646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a34070cd0 0x7f7a34157e40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400d5f0 tx=0x7f7a2400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.175+0000 7f7a29ffb640 1 -- 172.21.15.5:0/1850605646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a240125f0 con 0x7f7a34070cd0 2023-11-24T06:17:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.176+0000 7f7a29ffb640 1 -- 172.21.15.5:0/1850605646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a24012750 con 0x7f7a34070cd0 2023-11-24T06:17:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.176+0000 7f7a29ffb640 1 -- 172.21.15.5:0/1850605646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a24012b00 con 0x7f7a34070cd0 2023-11-24T06:17:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.176+0000 7f7a3b451640 1 -- 172.21.15.5:0/1850605646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a34159d40 con 0x7f7a34070cd0 2023-11-24T06:17:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.176+0000 7f7a3b451640 1 -- 172.21.15.5:0/1850605646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a341590e0 con 0x7f7a34070cd0 2023-11-24T06:17:04.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.177+0000 7f7a3b451640 1 -- 172.21.15.5:0/1850605646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a34159270 con 0x7f7a34070cd0 2023-11-24T06:17:04.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.178+0000 7f7a29ffb640 1 -- 172.21.15.5:0/1850605646 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a24012c60 con 0x7f7a34070cd0 2023-11-24T06:17:04.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.179+0000 7f7a29ffb640 1 --2- 172.21.15.5:0/1850605646 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7a1806ae40 0x7f7a1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:04.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.179+0000 7f7a29ffb640 1 -- 172.21.15.5:0/1850605646 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7a24089ae0 con 0x7f7a34070cd0 2023-11-24T06:17:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.179+0000 7f7a391ed640 1 --2- 172.21.15.5:0/1850605646 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7a1806ae40 0x7f7a1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:04.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.180+0000 7f7a391ed640 1 --2- 172.21.15.5:0/1850605646 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7a1806ae40 0x7f7a1806d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7a30003cc0 tx=0x7f7a300079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:04.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.183+0000 7f7a29ffb640 1 -- 172.21.15.5:0/1850605646 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a24054b90 con 0x7f7a34070cd0 2023-11-24T06:17:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.380+0000 7f7a3b451640 1 -- 172.21.15.5:0/1850605646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a34074770 con 0x7f7a34070cd0 2023-11-24T06:17:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.383+0000 7f7a29ffb640 1 -- 172.21.15.5:0/1850605646 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16658 (secure 0 0 0) 0x7f7a240581b0 con 0x7f7a34070cd0 2023-11-24T06:17:04.385 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:04.385 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:16:33.958286+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:15:49.850798+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T06:15:50.616892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.3c","mappings":[{"from":3,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:17:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.386+0000 7f7a3b451640 1 -- 172.21.15.5:0/1850605646 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7a1806ae40 msgr2=0x7f7a1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.386+0000 7f7a3b451640 1 --2- 172.21.15.5:0/1850605646 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7a1806ae40 0x7f7a1806d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7a30003cc0 tx=0x7f7a300079b0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.386+0000 7f7a3b451640 1 -- 172.21.15.5:0/1850605646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a34070cd0 msgr2=0x7f7a34157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.386+0000 7f7a3b451640 1 --2- 172.21.15.5:0/1850605646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a34070cd0 0x7f7a34157e40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7a2400d5f0 tx=0x7f7a2400dab0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.386+0000 7f7a3b451640 1 -- 172.21.15.5:0/1850605646 shutdown_connections 2023-11-24T06:17:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.386+0000 7f7a3b451640 1 --2- 172.21.15.5:0/1850605646 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7a1806ae40 0x7f7a1806d300 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.386+0000 7f7a3b451640 1 --2- 172.21.15.5:0/1850605646 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a34160970 0x7f7a3409c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.386+0000 7f7a3b451640 1 --2- 172.21.15.5:0/1850605646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a34070cd0 0x7f7a34157e40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.386+0000 7f7a3b451640 1 --2- 172.21.15.5:0/1850605646 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a34070780 0x7f7a3415c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.386+0000 7f7a3b451640 1 -- 172.21.15.5:0/1850605646 >> 172.21.15.5:0/1850605646 conn(0x7f7a340b0d80 msgr2=0x7f7a34161020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.386+0000 7f7a3b451640 1 -- 172.21.15.5:0/1850605646 shutdown_connections 2023-11-24T06:17:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.386+0000 7f7a3b451640 1 -- 172.21.15.5:0/1850605646 wait complete. 2023-11-24T06:17:04.410 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:17:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.634+0000 7f47a037c640 1 Processor -- start 2023-11-24T06:17:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.635+0000 7f47a037c640 1 -- start start 2023-11-24T06:17:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.636+0000 7f47a037c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f479815d460 0x7f47980cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.636+0000 7f47a037c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47980ceae0 con 0x7f479815d840 2023-11-24T06:17:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.637+0000 7f47a037c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47980d1c30 con 0x7f47980b64a0 2023-11-24T06:17:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.637+0000 7f47a037c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47980ce800 con 0x7f479815d460 2023-11-24T06:17:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.637+0000 7f479d917640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f479815d840 0x7f47980cdad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59100/0 (socket says 172.21.15.5:59100) 2023-11-24T06:17:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.637+0000 7f479d917640 1 -- 172.21.15.5:0/4037169238 learned_addr learned my addr 172.21.15.5:0/4037169238 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.637+0000 7f479e118640 1 --2- 172.21.15.5:0/4037169238 >> v2:172.21.15.6:3300/0 conn(0x7f479815d460 0x7f47980cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.637+0000 7f479d116640 1 -- 172.21.15.5:0/4037169238 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911543589 0 0) 0x7f4788001110 con 0x7f479815d840 2023-11-24T06:17:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.637+0000 7f479d116640 1 -- 172.21.15.5:0/4037169238 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4784003520 con 0x7f479815d840 2023-11-24T06:17:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.638+0000 7f479d116640 1 -- 172.21.15.5:0/4037169238 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3022414480 0 0) 0x7f4784003520 con 0x7f479815d840 2023-11-24T06:17:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.638+0000 7f479d116640 1 -- 172.21.15.5:0/4037169238 >> v2:172.21.15.6:3300/0 conn(0x7f479815d460 msgr2=0x7f47980cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.638+0000 7f479d116640 1 --2- 172.21.15.5:0/4037169238 >> v2:172.21.15.6:3300/0 conn(0x7f479815d460 0x7f47980cfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.638+0000 7f479d116640 1 -- 172.21.15.5:0/4037169238 >> v1:172.21.15.6:6789/0 conn(0x7f47980b64a0 legacy=0x7f47980d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.638+0000 7f479d116640 1 -- 172.21.15.5:0/4037169238 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980d15e0 con 0x7f479815d840 2023-11-24T06:17:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.638+0000 7f479d116640 1 -- 172.21.15.5:0/4037169238 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f4788001800 con 0x7f479815d840 2023-11-24T06:17:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.638+0000 7f479d116640 1 -- 172.21.15.5:0/4037169238 >> v1:172.21.15.5:6789/0 conn(0x7f479815d840 legacy=0x7f47980cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.638+0000 7f479d116640 1 --2- 172.21.15.5:0/4037169238 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f47840039c0 0x7f4784003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.638+0000 7f479d116640 1 --2- 172.21.15.5:0/4037169238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47840047c0 0x7f4784004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.638+0000 7f479d116640 1 --2- 172.21.15.5:0/4037169238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47840051d0 0x7f478400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.638+0000 7f479d116640 1 -- 172.21.15.5:0/4037169238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47980ce800 con 0x7f47840051d0 2023-11-24T06:17:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.638+0000 7f479d116640 1 -- 172.21.15.5:0/4037169238 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478400c120 con 0x7f47840047c0 2023-11-24T06:17:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.638+0000 7f479d116640 1 -- 172.21.15.5:0/4037169238 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478400c2e0 con 0x7f47840039c0 2023-11-24T06:17:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.638+0000 7f479e919640 1 --2- 172.21.15.5:0/4037169238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47840051d0 0x7f478400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.639+0000 7f479d917640 1 --2- 172.21.15.5:0/4037169238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47840047c0 0x7f4784004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.639+0000 7f479e118640 1 --2- 172.21.15.5:0/4037169238 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f47840039c0 0x7f4784003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.639+0000 7f479e919640 1 -- 172.21.15.5:0/4037169238 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f47840039c0 msgr2=0x7f4784003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.639+0000 7f479e919640 1 --2- 172.21.15.5:0/4037169238 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f47840039c0 0x7f4784003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.639+0000 7f479e919640 1 -- 172.21.15.5:0/4037169238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47840047c0 msgr2=0x7f4784004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.639+0000 7f479e919640 1 --2- 172.21.15.5:0/4037169238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47840047c0 0x7f4784004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.639+0000 7f479e919640 1 -- 172.21.15.5:0/4037169238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4784003520 con 0x7f47840051d0 2023-11-24T06:17:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.639+0000 7f479d917640 1 --2- 172.21.15.5:0/4037169238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f47840047c0 0x7f4784004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.640+0000 7f479e919640 1 -- 172.21.15.5:0/4037169238 reap_dead start 2023-11-24T06:17:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.640+0000 7f479e919640 1 --2- 172.21.15.5:0/4037169238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47840051d0 0x7f478400b980 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f47980b68a0 tx=0x7f47840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fda7fbee48902b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.640+0000 7f479d116640 1 -- 172.21.15.5:0/4037169238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4790003b70 con 0x7f47840051d0 2023-11-24T06:17:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.640+0000 7f479d116640 1 -- 172.21.15.5:0/4037169238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4790003cd0 con 0x7f47840051d0 2023-11-24T06:17:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.640+0000 7f479d116640 1 -- 172.21.15.5:0/4037169238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4790005310 con 0x7f47840051d0 2023-11-24T06:17:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.641+0000 7f47a037c640 1 -- 172.21.15.5:0/4037169238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47840051d0 msgr2=0x7f478400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.641+0000 7f47a037c640 1 --2- 172.21.15.5:0/4037169238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47840051d0 0x7f478400b980 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f47980b68a0 tx=0x7f47840057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.641+0000 7f47a037c640 1 -- 172.21.15.5:0/4037169238 shutdown_connections 2023-11-24T06:17:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.641+0000 7f47a037c640 1 --2- 172.21.15.5:0/4037169238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47840051d0 0x7f478400b980 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.641+0000 7f47a037c640 1 -- 172.21.15.5:0/4037169238 >> 172.21.15.5:0/4037169238 conn(0x7f479806b420 msgr2=0x7f479806b830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.641+0000 7f47a037c640 1 -- 172.21.15.5:0/4037169238 shutdown_connections 2023-11-24T06:17:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.641+0000 7f47a037c640 1 -- 172.21.15.5:0/4037169238 wait complete. 2023-11-24T06:17:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.643+0000 7f47a037c640 1 Processor -- start 2023-11-24T06:17:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.643+0000 7f47a037c640 1 -- start start 2023-11-24T06:17:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.644+0000 7f47a037c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f47840051d0 0x7f4798149d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.644+0000 7f47a037c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47980a9fb0 0x7f479814c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.644+0000 7f47a037c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f479814b3f0 0x7f479815a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.644+0000 7f47a037c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479814a350 con 0x7f47980a9fb0 2023-11-24T06:17:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.644+0000 7f47a037c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479814a630 con 0x7f479814b3f0 2023-11-24T06:17:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.644+0000 7f47a037c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479814a910 con 0x7f47840051d0 2023-11-24T06:17:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.644+0000 7f479d917640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47980a9fb0 0x7f479814c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.644+0000 7f479d917640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47980a9fb0 0x7f479814c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53478/0 (socket says 172.21.15.5:53478) 2023-11-24T06:17:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.644+0000 7f479d917640 1 -- 172.21.15.5:0/3747749018 learned_addr learned my addr 172.21.15.5:0/3747749018 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.645+0000 7f479d917640 1 -- 172.21.15.5:0/3747749018 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f47840051d0 msgr2=0x7f4798149d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.645+0000 7f479d917640 1 --2- 172.21.15.5:0/3747749018 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f47840051d0 0x7f4798149d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.645+0000 7f479d917640 1 -- 172.21.15.5:0/3747749018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f479814b3f0 msgr2=0x7f479815a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.645+0000 7f479d917640 1 --2- 172.21.15.5:0/3747749018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f479814b3f0 0x7f479815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.645+0000 7f479d917640 1 -- 172.21.15.5:0/3747749018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479814abf0 con 0x7f47980a9fb0 2023-11-24T06:17:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.645+0000 7f479d917640 1 --2- 172.21.15.5:0/3747749018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47980a9fb0 0x7f479814c000 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f478800d770 tx=0x7f478800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.645+0000 7f478effd640 1 -- 172.21.15.5:0/3747749018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47880074e0 con 0x7f47980a9fb0 2023-11-24T06:17:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.645+0000 7f478effd640 1 -- 172.21.15.5:0/3747749018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4788007640 con 0x7f47980a9fb0 2023-11-24T06:17:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.645+0000 7f478effd640 1 -- 172.21.15.5:0/3747749018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788004a00 con 0x7f47980a9fb0 2023-11-24T06:17:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.645+0000 7f47a037c640 1 -- 172.21.15.5:0/3747749018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479814aed0 con 0x7f47980a9fb0 2023-11-24T06:17:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.646+0000 7f47a037c640 1 -- 172.21.15.5:0/3747749018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479814c540 con 0x7f47980a9fb0 2023-11-24T06:17:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.647+0000 7f47a037c640 1 -- 172.21.15.5:0/3747749018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4798074770 con 0x7f47980a9fb0 2023-11-24T06:17:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.648+0000 7f478effd640 1 -- 172.21.15.5:0/3747749018 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4788004b60 con 0x7f47980a9fb0 2023-11-24T06:17:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.649+0000 7f478effd640 1 --2- 172.21.15.5:0/3747749018 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f477c06ae70 0x7f477c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.649+0000 7f478effd640 1 -- 172.21.15.5:0/3747749018 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4788088ed0 con 0x7f47980a9fb0 2023-11-24T06:17:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.649+0000 7f479e118640 1 --2- 172.21.15.5:0/3747749018 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f477c06ae70 0x7f477c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.650+0000 7f479e118640 1 --2- 172.21.15.5:0/3747749018 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f477c06ae70 0x7f477c06d330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4794009950 tx=0x7f4794008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.653+0000 7f478effd640 1 -- 172.21.15.5:0/3747749018 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4788053e60 con 0x7f47980a9fb0 2023-11-24T06:17:04.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.883+0000 7f47a037c640 1 -- 172.21.15.5:0/3747749018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f47980ceae0 con 0x7f47980a9fb0 2023-11-24T06:17:04.887 INFO:tasks.ceph.mon.a.smithi005.stderr:2023-11-24T06:17:04.885+0000 7f463fba2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:17:04.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.889+0000 7f478effd640 1 -- 172.21.15.5:0/3747749018 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f47980ceae0 con 0x7f47980a9fb0 2023-11-24T06:17:04.891 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:17:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.892+0000 7f47a037c640 1 -- 172.21.15.5:0/3747749018 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f477c06ae70 msgr2=0x7f477c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.892+0000 7f47a037c640 1 --2- 172.21.15.5:0/3747749018 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f477c06ae70 0x7f477c06d330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4794009950 tx=0x7f4794008040 comp rx=0 tx=0).stop 2023-11-24T06:17:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.892+0000 7f47a037c640 1 -- 172.21.15.5:0/3747749018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47980a9fb0 msgr2=0x7f479814c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.892+0000 7f47a037c640 1 --2- 172.21.15.5:0/3747749018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47980a9fb0 0x7f479814c000 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f478800d770 tx=0x7f478800dc30 comp rx=0 tx=0).stop 2023-11-24T06:17:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.892+0000 7f47a037c640 1 -- 172.21.15.5:0/3747749018 shutdown_connections 2023-11-24T06:17:04.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.892+0000 7f47a037c640 1 --2- 172.21.15.5:0/3747749018 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f477c06ae70 0x7f477c06d330 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.892+0000 7f47a037c640 1 --2- 172.21.15.5:0/3747749018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f479814b3f0 0x7f479815a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.892+0000 7f47a037c640 1 --2- 172.21.15.5:0/3747749018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47980a9fb0 0x7f479814c000 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.892+0000 7f47a037c640 1 --2- 172.21.15.5:0/3747749018 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f47840051d0 0x7f4798149d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:04.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.892+0000 7f47a037c640 1 -- 172.21.15.5:0/3747749018 >> 172.21.15.5:0/3747749018 conn(0x7f479806b420 msgr2=0x7f47980d3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:04.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.892+0000 7f47a037c640 1 -- 172.21.15.5:0/3747749018 shutdown_connections 2023-11-24T06:17:04.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:04.893+0000 7f47a037c640 1 -- 172.21.15.5:0/3747749018 wait complete. 2023-11-24T06:17:04.915 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:17:04.923 INFO:tasks.ceph.mds.b.smithi006.stderr: -3921> 2023-11-24T06:15:57.776+0000 7f9a7f223640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T06:17:05.003 INFO:tasks.ceph.mds.b.smithi006.stdout:starting mds.b at 2023-11-24T06:17:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.137+0000 7f1448c7e640 1 Processor -- start 2023-11-24T06:17:05.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.137+0000 7f1448c7e640 1 -- start start 2023-11-24T06:17:05.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.139+0000 7f1448c7e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f14441612c0 0x7f14440709d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:05.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.139+0000 7f1448c7e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14441616a0 0x7f144415d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:05.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.139+0000 7f1448c7e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1444070f10 con 0x7f144415dd70 2023-11-24T06:17:05.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.139+0000 7f1448c7e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144415e280 con 0x7f14441612c0 2023-11-24T06:17:05.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.139+0000 7f1448c7e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144415e560 con 0x7f14441616a0 2023-11-24T06:17:05.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.140+0000 7f14437fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f14441612c0 0x7f14440709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:05.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.140+0000 7f14437fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f14441612c0 0x7f14440709d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53494/0 (socket says 172.21.15.5:53494) 2023-11-24T06:17:05.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.140+0000 7f14437fe640 1 -- 172.21.15.5:0/2796613432 learned_addr learned my addr 172.21.15.5:0/2796613432 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:05.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.140+0000 7f1442ffd640 1 --2- 172.21.15.5:0/2796613432 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14441616a0 0x7f144415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.140+0000 7f14437fe640 1 -- 172.21.15.5:0/2796613432 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14441616a0 msgr2=0x7f144415d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.140+0000 7f14437fe640 1 --2- 172.21.15.5:0/2796613432 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14441616a0 0x7f144415d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.140+0000 7f14437fe640 1 -- 172.21.15.5:0/2796613432 >> v1:172.21.15.6:6789/0 conn(0x7f144415dd70 legacy=0x7f1444071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.140+0000 7f14437fe640 1 -- 172.21.15.5:0/2796613432 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14440701d0 con 0x7f14441612c0 2023-11-24T06:17:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.142+0000 7f14437fe640 1 --2- 172.21.15.5:0/2796613432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14441612c0 0x7f14440709d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1434009e50 tx=0x7f143402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bf1b294026dc10 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:05.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.142+0000 7f14427fc640 1 -- 172.21.15.5:0/2796613432 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143402ca70 con 0x7f14441612c0 2023-11-24T06:17:05.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.142+0000 7f14427fc640 1 -- 172.21.15.5:0/2796613432 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143402cbd0 con 0x7f14441612c0 2023-11-24T06:17:05.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.142+0000 7f14427fc640 1 -- 172.21.15.5:0/2796613432 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14340343f0 con 0x7f14441612c0 2023-11-24T06:17:05.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.143+0000 7f1448c7e640 1 -- 172.21.15.5:0/2796613432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14441612c0 msgr2=0x7f14440709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:05.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.143+0000 7f1448c7e640 1 --2- 172.21.15.5:0/2796613432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14441612c0 0x7f14440709d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1434009e50 tx=0x7f143402bed0 comp rx=0 tx=0).stop 2023-11-24T06:17:05.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.143+0000 7f1448c7e640 1 -- 172.21.15.5:0/2796613432 shutdown_connections 2023-11-24T06:17:05.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.143+0000 7f1448c7e640 1 --2- 172.21.15.5:0/2796613432 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14441616a0 0x7f144415d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:05.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.143+0000 7f1448c7e640 1 --2- 172.21.15.5:0/2796613432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14441612c0 0x7f14440709d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:05.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.143+0000 7f1448c7e640 1 -- 172.21.15.5:0/2796613432 >> 172.21.15.5:0/2796613432 conn(0x7f144406ae50 msgr2=0x7f14440cec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:05.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.143+0000 7f1448c7e640 1 -- 172.21.15.5:0/2796613432 shutdown_connections 2023-11-24T06:17:05.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.143+0000 7f1448c7e640 1 -- 172.21.15.5:0/2796613432 wait complete. 2023-11-24T06:17:05.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.145+0000 7f1448c7e640 1 Processor -- start 2023-11-24T06:17:05.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.145+0000 7f1448c7e640 1 -- start start 2023-11-24T06:17:05.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.146+0000 7f1448c7e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f144415dd70 0x7f1444151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:05.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.146+0000 7f1448c7e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14441612c0 0x7f14440b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:05.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.146+0000 7f1448c7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14441616a0 0x7f144415c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:05.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.146+0000 7f1448c7e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144414fee0 con 0x7f14441616a0 2023-11-24T06:17:05.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.146+0000 7f1448c7e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14440b0d70 con 0x7f144415dd70 2023-11-24T06:17:05.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.146+0000 7f1448c7e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1444151e40 con 0x7f14441612c0 2023-11-24T06:17:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.146+0000 7f1443fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14441616a0 0x7f144415c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.146+0000 7f1443fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14441616a0 0x7f144415c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53510/0 (socket says 172.21.15.5:53510) 2023-11-24T06:17:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.146+0000 7f1443fff640 1 -- 172.21.15.5:0/540376675 learned_addr learned my addr 172.21.15.5:0/540376675 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.147+0000 7f1442ffd640 1 --2- 172.21.15.5:0/540376675 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14441612c0 0x7f14440b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.147+0000 7f1443fff640 1 -- 172.21.15.5:0/540376675 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14441612c0 msgr2=0x7f14440b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.147+0000 7f1443fff640 1 --2- 172.21.15.5:0/540376675 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14441612c0 0x7f14440b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.147+0000 7f1443fff640 1 -- 172.21.15.5:0/540376675 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f144415dd70 msgr2=0x7f1444151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.147+0000 7f1443fff640 1 --2- 172.21.15.5:0/540376675 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f144415dd70 0x7f1444151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.147+0000 7f1443fff640 1 -- 172.21.15.5:0/540376675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1434009b30 con 0x7f14441616a0 2023-11-24T06:17:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.147+0000 7f1443fff640 1 --2- 172.21.15.5:0/540376675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14441616a0 0x7f144415c450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b440 tx=0x7f143c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.147+0000 7f142bfff640 1 -- 172.21.15.5:0/540376675 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c011830 con 0x7f14441616a0 2023-11-24T06:17:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.147+0000 7f142bfff640 1 -- 172.21.15.5:0/540376675 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c011990 con 0x7f14441616a0 2023-11-24T06:17:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.147+0000 7f142bfff640 1 -- 172.21.15.5:0/540376675 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c011d40 con 0x7f14441616a0 2023-11-24T06:17:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.147+0000 7f1448c7e640 1 -- 172.21.15.5:0/540376675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14440b0160 con 0x7f14441616a0 2023-11-24T06:17:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.148+0000 7f1448c7e640 1 -- 172.21.15.5:0/540376675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14440b0720 con 0x7f14441616a0 2023-11-24T06:17:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.149+0000 7f1448c7e640 1 -- 172.21.15.5:0/540376675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144404e840 con 0x7f14441616a0 2023-11-24T06:17:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.150+0000 7f142bfff640 1 -- 172.21.15.5:0/540376675 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f143c01b460 con 0x7f14441616a0 2023-11-24T06:17:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.151+0000 7f142bfff640 1 --2- 172.21.15.5:0/540376675 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f143006ae40 0x7f143006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.152+0000 7f142bfff640 1 -- 172.21.15.5:0/540376675 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f143c0890e0 con 0x7f14441616a0 2023-11-24T06:17:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.152+0000 7f14437fe640 1 --2- 172.21.15.5:0/540376675 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f143006ae40 0x7f143006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.152+0000 7f14437fe640 1 --2- 172.21.15.5:0/540376675 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f143006ae40 0x7f143006d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1434035000 tx=0x7f143402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:05.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.156+0000 7f142bfff640 1 -- 172.21.15.5:0/540376675 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f143c054130 con 0x7f14441616a0 2023-11-24T06:17:05.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.392+0000 7f1448c7e640 1 -- 172.21.15.5:0/540376675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f14440b0eb0 con 0x7f14441616a0 2023-11-24T06:17:05.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.892+0000 7f142bfff640 1 -- 172.21.15.5:0/540376675 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f143c057750 con 0x7f14441616a0 2023-11-24T06:17:05.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.896+0000 7f1448c7e640 1 -- 172.21.15.5:0/540376675 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f143006ae40 msgr2=0x7f143006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:05.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.896+0000 7f1448c7e640 1 --2- 172.21.15.5:0/540376675 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f143006ae40 0x7f143006d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1434035000 tx=0x7f143402e000 comp rx=0 tx=0).stop 2023-11-24T06:17:05.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.896+0000 7f1448c7e640 1 -- 172.21.15.5:0/540376675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14441616a0 msgr2=0x7f144415c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:05.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.896+0000 7f1448c7e640 1 --2- 172.21.15.5:0/540376675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14441616a0 0x7f144415c450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b440 tx=0x7f143c00b900 comp rx=0 tx=0).stop 2023-11-24T06:17:05.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.896+0000 7f1448c7e640 1 -- 172.21.15.5:0/540376675 shutdown_connections 2023-11-24T06:17:05.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.897+0000 7f1448c7e640 1 --2- 172.21.15.5:0/540376675 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f143006ae40 0x7f143006d300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:05.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.897+0000 7f1448c7e640 1 --2- 172.21.15.5:0/540376675 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14441612c0 0x7f14440b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:05.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.897+0000 7f1448c7e640 1 --2- 172.21.15.5:0/540376675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14441616a0 0x7f144415c450 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:05.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.897+0000 7f1448c7e640 1 --2- 172.21.15.5:0/540376675 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f144415dd70 0x7f1444151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:05.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.897+0000 7f1448c7e640 1 -- 172.21.15.5:0/540376675 >> 172.21.15.5:0/540376675 conn(0x7f144406ae50 msgr2=0x7f14440b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:05.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.897+0000 7f1448c7e640 1 -- 172.21.15.5:0/540376675 shutdown_connections 2023-11-24T06:17:05.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:05.897+0000 7f1448c7e640 1 -- 172.21.15.5:0/540376675 wait complete. 2023-11-24T06:17:05.920 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:17:06.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.151+0000 7f803359e640 1 Processor -- start 2023-11-24T06:17:06.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.151+0000 7f803359e640 1 -- start start 2023-11-24T06:17:06.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.153+0000 7f803359e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f803415d340 0x7f80340d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:06.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.153+0000 7f803359e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80340cef30 con 0x7f80340cdbb0 2023-11-24T06:17:06.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.153+0000 7f803359e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80340d11a0 con 0x7f80340d1d90 2023-11-24T06:17:06.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.153+0000 7f803359e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80340ce320 con 0x7f803415d340 2023-11-24T06:17:06.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.153+0000 7f803259c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f803415d340 0x7f80340d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:06.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.153+0000 7f802bfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f80340cdbb0 0x7f80340d1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59104/0 (socket says 172.21.15.5:59104) 2023-11-24T06:17:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.153+0000 7f802bfff640 1 -- 172.21.15.5:0/1870517599 learned_addr learned my addr 172.21.15.5:0/1870517599 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.154+0000 7f8031d9b640 1 -- 172.21.15.5:0/1870517599 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114835686 0 0) 0x7f80340cef30 con 0x7f80340cdbb0 2023-11-24T06:17:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.154+0000 7f803259c640 1 -- 172.21.15.5:0/1870517599 >> v1:172.21.15.6:6789/0 conn(0x7f80340d1d90 legacy=0x7f80340b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.154+0000 7f803259c640 1 -- 172.21.15.5:0/1870517599 >> v1:172.21.15.5:6789/0 conn(0x7f80340cdbb0 legacy=0x7f80340d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.154+0000 7f803259c640 1 -- 172.21.15.5:0/1870517599 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80340ce600 con 0x7f803415d340 2023-11-24T06:17:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.155+0000 7f803259c640 1 --2- 172.21.15.5:0/1870517599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f803415d340 0x7f80340d00b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f80240097a0 tx=0x7f802402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823b748a11d5e75 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.155+0000 7f8031d9b640 1 -- 172.21.15.5:0/1870517599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024031830 con 0x7f803415d340 2023-11-24T06:17:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.155+0000 7f8031d9b640 1 -- 172.21.15.5:0/1870517599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8024031990 con 0x7f803415d340 2023-11-24T06:17:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.155+0000 7f8031d9b640 1 -- 172.21.15.5:0/1870517599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024031d20 con 0x7f803415d340 2023-11-24T06:17:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.156+0000 7f803359e640 1 -- 172.21.15.5:0/1870517599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f803415d340 msgr2=0x7f80340d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.156+0000 7f803359e640 1 --2- 172.21.15.5:0/1870517599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f803415d340 0x7f80340d00b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f80240097a0 tx=0x7f802402b8d0 comp rx=0 tx=0).stop 2023-11-24T06:17:06.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.156+0000 7f803359e640 1 -- 172.21.15.5:0/1870517599 shutdown_connections 2023-11-24T06:17:06.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.156+0000 7f803359e640 1 --2- 172.21.15.5:0/1870517599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f803415d340 0x7f80340d00b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:06.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.156+0000 7f803359e640 1 -- 172.21.15.5:0/1870517599 >> 172.21.15.5:0/1870517599 conn(0x7f803409c370 msgr2=0x7f80340d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:06.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.156+0000 7f803359e640 1 -- 172.21.15.5:0/1870517599 shutdown_connections 2023-11-24T06:17:06.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.156+0000 7f803359e640 1 -- 172.21.15.5:0/1870517599 wait complete. 2023-11-24T06:17:06.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.158+0000 7f803359e640 1 Processor -- start 2023-11-24T06:17:06.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.158+0000 7f803359e640 1 -- start start 2023-11-24T06:17:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.159+0000 7f803359e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80340cdbb0 0x7f803415ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.159+0000 7f803359e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80340d1d90 0x7f80340b8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.159+0000 7f803359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f803415d340 0x7f8034146260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.159+0000 7f803359e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803415f520 con 0x7f803415d340 2023-11-24T06:17:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.159+0000 7f803359e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80340c7090 con 0x7f80340cdbb0 2023-11-24T06:17:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.159+0000 7f803359e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803415f7c0 con 0x7f80340d1d90 2023-11-24T06:17:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.160+0000 7f8032d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f803415d340 0x7f8034146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.160+0000 7f8032d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f803415d340 0x7f8034146260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53528/0 (socket says 172.21.15.5:53528) 2023-11-24T06:17:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.160+0000 7f8032d9d640 1 -- 172.21.15.5:0/3189873189 learned_addr learned my addr 172.21.15.5:0/3189873189 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.160+0000 7f803259c640 1 --2- 172.21.15.5:0/3189873189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80340cdbb0 0x7f803415ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.160+0000 7f802bfff640 1 --2- 172.21.15.5:0/3189873189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80340d1d90 0x7f80340b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.160+0000 7f8032d9d640 1 -- 172.21.15.5:0/3189873189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80340d1d90 msgr2=0x7f80340b8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.160+0000 7f8032d9d640 1 --2- 172.21.15.5:0/3189873189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80340d1d90 0x7f80340b8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.160+0000 7f8032d9d640 1 -- 172.21.15.5:0/3189873189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80340cdbb0 msgr2=0x7f803415ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.160+0000 7f8032d9d640 1 --2- 172.21.15.5:0/3189873189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80340cdbb0 0x7f803415ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.160+0000 7f8032d9d640 1 -- 172.21.15.5:0/3189873189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8024009400 con 0x7f803415d340 2023-11-24T06:17:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.161+0000 7f8032d9d640 1 --2- 172.21.15.5:0/3189873189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f803415d340 0x7f8034146260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f802c00c700 tx=0x7f802c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.161+0000 7f802affd640 1 -- 172.21.15.5:0/3189873189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c00aa50 con 0x7f803415d340 2023-11-24T06:17:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.161+0000 7f802affd640 1 -- 172.21.15.5:0/3189873189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c00abb0 con 0x7f803415d340 2023-11-24T06:17:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.161+0000 7f802affd640 1 -- 172.21.15.5:0/3189873189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c00ce10 con 0x7f803415d340 2023-11-24T06:17:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.161+0000 7f803359e640 1 -- 172.21.15.5:0/3189873189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803415faa0 con 0x7f803415d340 2023-11-24T06:17:06.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.161+0000 7f803359e640 1 -- 172.21.15.5:0/3189873189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80340c7370 con 0x7f803415d340 2023-11-24T06:17:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.162+0000 7f803359e640 1 -- 172.21.15.5:0/3189873189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803415fce0 con 0x7f803415d340 2023-11-24T06:17:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.163+0000 7f802affd640 1 -- 172.21.15.5:0/3189873189 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f802c012420 con 0x7f803415d340 2023-11-24T06:17:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.164+0000 7f802affd640 1 --2- 172.21.15.5:0/3189873189 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f801406ae40 0x7f801406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.164+0000 7f802affd640 1 -- 172.21.15.5:0/3189873189 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f802c089b20 con 0x7f803415d340 2023-11-24T06:17:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.164+0000 7f803259c640 1 --2- 172.21.15.5:0/3189873189 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f801406ae40 0x7f801406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:06.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.165+0000 7f803259c640 1 --2- 172.21.15.5:0/3189873189 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f801406ae40 0x7f801406d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8024036000 tx=0x7f802403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:06.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.168+0000 7f802affd640 1 -- 172.21.15.5:0/3189873189 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f803415fce0 con 0x7f803415d340 2023-11-24T06:17:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:06.361+0000 7f803359e640 1 -- 172.21.15.5:0/3189873189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f803415fce0 con 0x7f803415d340 2023-11-24T06:17:07.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.307+0000 7f802affd640 1 -- 172.21.15.5:0/3189873189 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7f803415fce0 con 0x7f803415d340 2023-11-24T06:17:07.310 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:17:07.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.312+0000 7f803359e640 1 -- 172.21.15.5:0/3189873189 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f801406ae40 msgr2=0x7f801406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:07.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.312+0000 7f803359e640 1 --2- 172.21.15.5:0/3189873189 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f801406ae40 0x7f801406d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8024036000 tx=0x7f802403c040 comp rx=0 tx=0).stop 2023-11-24T06:17:07.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.312+0000 7f803359e640 1 -- 172.21.15.5:0/3189873189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f803415d340 msgr2=0x7f8034146260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:07.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.312+0000 7f803359e640 1 --2- 172.21.15.5:0/3189873189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f803415d340 0x7f8034146260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f802c00c700 tx=0x7f802c00cbc0 comp rx=0 tx=0).stop 2023-11-24T06:17:07.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.313+0000 7f803359e640 1 -- 172.21.15.5:0/3189873189 shutdown_connections 2023-11-24T06:17:07.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.313+0000 7f803359e640 1 --2- 172.21.15.5:0/3189873189 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f801406ae40 0x7f801406d300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:07.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.313+0000 7f803359e640 1 --2- 172.21.15.5:0/3189873189 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f80340d1d90 0x7f80340b8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:07.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.313+0000 7f803359e640 1 --2- 172.21.15.5:0/3189873189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f803415d340 0x7f8034146260 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:07.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.313+0000 7f803359e640 1 --2- 172.21.15.5:0/3189873189 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f80340cdbb0 0x7f803415ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:07.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.313+0000 7f803359e640 1 -- 172.21.15.5:0/3189873189 >> 172.21.15.5:0/3189873189 conn(0x7f803409c370 msgr2=0x7f80340b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:07.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.314+0000 7f803359e640 1 -- 172.21.15.5:0/3189873189 shutdown_connections 2023-11-24T06:17:07.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.314+0000 7f803359e640 1 -- 172.21.15.5:0/3189873189 wait complete. 2023-11-24T06:17:07.338 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:17:07.412 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:17:07.417 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:17:07.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.540+0000 7ff08163d640 1 Processor -- start 2023-11-24T06:17:07.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.540+0000 7ff08163d640 1 -- start start 2023-11-24T06:17:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.542+0000 7ff08163d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff07c0ae270 0x7ff07c0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.542+0000 7ff08163d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07c15b730 con 0x7ff07c15b350 2023-11-24T06:17:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.542+0000 7ff08163d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07c0ac740 con 0x7ff07c09cb30 2023-11-24T06:17:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.542+0000 7ff08163d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff07c0ac460 con 0x7ff07c0ae270 2023-11-24T06:17:07.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.542+0000 7ff07affd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff07c15b350 0x7ff07c0b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59120/0 (socket says 172.21.15.5:59120) 2023-11-24T06:17:07.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.542+0000 7ff07affd640 1 -- 172.21.15.5:0/2602706015 learned_addr learned my addr 172.21.15.5:0/2602706015 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:07.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.542+0000 7ff07b7fe640 1 --2- 172.21.15.5:0/2602706015 >> v2:172.21.15.5:3300/0 conn(0x7ff07c0ae270 0x7ff07c0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:07.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.543+0000 7ff079ffb640 1 -- 172.21.15.5:0/2602706015 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3558470157 0 0) 0x7ff06c001110 con 0x7ff07c15b350 2023-11-24T06:17:07.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.543+0000 7ff079ffb640 1 -- 172.21.15.5:0/2602706015 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff064003520 con 0x7ff07c15b350 2023-11-24T06:17:07.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.543+0000 7ff07b7fe640 1 -- 172.21.15.5:0/2602706015 >> v1:172.21.15.6:6789/0 conn(0x7ff07c09cb30 legacy=0x7ff07c0adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:07.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.543+0000 7ff07b7fe640 1 -- 172.21.15.5:0/2602706015 >> v1:172.21.15.5:6789/0 conn(0x7ff07c15b350 legacy=0x7ff07c0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:07.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.543+0000 7ff07b7fe640 1 -- 172.21.15.5:0/2602706015 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c0b2a50 con 0x7ff07c0ae270 2023-11-24T06:17:07.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.544+0000 7ff07b7fe640 1 --2- 172.21.15.5:0/2602706015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07c0ae270 0x7ff07c0abe80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff074009d90 tx=0x7ff07402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72f1245a9f83564 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:07.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.544+0000 7ff079ffb640 1 -- 172.21.15.5:0/2602706015 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07402cd00 con 0x7ff07c0ae270 2023-11-24T06:17:07.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.544+0000 7ff079ffb640 1 -- 172.21.15.5:0/2602706015 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07402ce60 con 0x7ff07c0ae270 2023-11-24T06:17:07.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.544+0000 7ff079ffb640 1 -- 172.21.15.5:0/2602706015 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074034660 con 0x7ff07c0ae270 2023-11-24T06:17:07.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.545+0000 7ff08163d640 1 -- 172.21.15.5:0/2602706015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07c0ae270 msgr2=0x7ff07c0abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:07.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.545+0000 7ff08163d640 1 --2- 172.21.15.5:0/2602706015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07c0ae270 0x7ff07c0abe80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff074009d90 tx=0x7ff07402bf50 comp rx=0 tx=0).stop 2023-11-24T06:17:07.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.545+0000 7ff08163d640 1 -- 172.21.15.5:0/2602706015 shutdown_connections 2023-11-24T06:17:07.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.545+0000 7ff08163d640 1 --2- 172.21.15.5:0/2602706015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07c0ae270 0x7ff07c0abe80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:07.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.545+0000 7ff08163d640 1 -- 172.21.15.5:0/2602706015 >> 172.21.15.5:0/2602706015 conn(0x7ff07c0b0ba0 msgr2=0x7ff07c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.545+0000 7ff08163d640 1 -- 172.21.15.5:0/2602706015 shutdown_connections 2023-11-24T06:17:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.545+0000 7ff08163d640 1 -- 172.21.15.5:0/2602706015 wait complete. 2023-11-24T06:17:07.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.547+0000 7ff08163d640 1 Processor -- start 2023-11-24T06:17:07.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.547+0000 7ff08163d640 1 -- start start 2023-11-24T06:17:07.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.548+0000 7ff08163d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07c09cb30 0x7ff07c13e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:07.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.548+0000 7ff08163d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff07c0ae270 0x7ff07c159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:07.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.548+0000 7ff08163d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff07c15b350 0x7ff07c1409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:07.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.548+0000 7ff08163d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c15d080 con 0x7ff07c09cb30 2023-11-24T06:17:07.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.548+0000 7ff08163d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c13e690 con 0x7ff07c0ae270 2023-11-24T06:17:07.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.548+0000 7ff08163d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c156c60 con 0x7ff07c15b350 2023-11-24T06:17:07.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.548+0000 7ff07affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07c09cb30 0x7ff07c13e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:07.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.548+0000 7ff07affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07c09cb30 0x7ff07c13e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53540/0 (socket says 172.21.15.5:53540) 2023-11-24T06:17:07.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.548+0000 7ff07affd640 1 -- 172.21.15.5:0/1045766048 learned_addr learned my addr 172.21.15.5:0/1045766048 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:07.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.549+0000 7ff07b7fe640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff07c15b350 0x7ff07c1409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:07.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.549+0000 7ff07a7fc640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff07c0ae270 0x7ff07c159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:07.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.549+0000 7ff07affd640 1 -- 172.21.15.5:0/1045766048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff07c15b350 msgr2=0x7ff07c1409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:07.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.549+0000 7ff07affd640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff07c15b350 0x7ff07c1409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:07.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.549+0000 7ff07affd640 1 -- 172.21.15.5:0/1045766048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff07c0ae270 msgr2=0x7ff07c159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:07.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.549+0000 7ff07affd640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff07c0ae270 0x7ff07c159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:07.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.549+0000 7ff07affd640 1 -- 172.21.15.5:0/1045766048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff074009a70 con 0x7ff07c09cb30 2023-11-24T06:17:07.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.549+0000 7ff07b7fe640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff07c15b350 0x7ff07c1409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:07.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.549+0000 7ff07a7fc640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff07c0ae270 0x7ff07c159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:07.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.549+0000 7ff07affd640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07c09cb30 0x7ff07c13e150 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00c420 tx=0x7ff06c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:07.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.549+0000 7ff0637fe640 1 -- 172.21.15.5:0/1045766048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c005ce0 con 0x7ff07c09cb30 2023-11-24T06:17:07.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.550+0000 7ff0637fe640 1 -- 172.21.15.5:0/1045766048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c005e40 con 0x7ff07c09cb30 2023-11-24T06:17:07.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.550+0000 7ff08163d640 1 -- 172.21.15.5:0/1045766048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07c156980 con 0x7ff07c09cb30 2023-11-24T06:17:07.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.550+0000 7ff0637fe640 1 -- 172.21.15.5:0/1045766048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c0072f0 con 0x7ff07c09cb30 2023-11-24T06:17:07.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.550+0000 7ff08163d640 1 -- 172.21.15.5:0/1045766048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07c15cca0 con 0x7ff07c09cb30 2023-11-24T06:17:07.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.551+0000 7ff08163d640 1 -- 172.21.15.5:0/1045766048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff07c15d1c0 con 0x7ff07c09cb30 2023-11-24T06:17:07.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.552+0000 7ff0637fe640 1 -- 172.21.15.5:0/1045766048 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff06c007510 con 0x7ff07c09cb30 2023-11-24T06:17:07.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.553+0000 7ff0637fe640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff06806ae40 0x7ff06806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:07.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.553+0000 7ff0637fe640 1 -- 172.21.15.5:0/1045766048 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5502+0+0 (secure 0 0 0) 0x7ff06c0889c0 con 0x7ff07c09cb30 2023-11-24T06:17:07.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.553+0000 7ff07a7fc640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff06806ae40 0x7ff06806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:07.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.554+0000 7ff07a7fc640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff06806ae40 0x7ff06806d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0cf700 tx=0x7ff070009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:07.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.558+0000 7ff0637fe640 1 -- 172.21.15.5:0/1045766048 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff06c053b30 con 0x7ff07c09cb30 2023-11-24T06:17:07.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:07.750+0000 7ff08163d640 1 -- 172.21.15.5:0/1045766048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff07c15b730 con 0x7ff07c09cb30 2023-11-24T06:17:08.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.311+0000 7ff0637fe640 1 -- 172.21.15.5:0/1045766048 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff07c15b730 con 0x7ff07c09cb30 2023-11-24T06:17:08.313 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2023-11-24T06:17:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.314+0000 7ff08163d640 1 -- 172.21.15.5:0/1045766048 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff06806ae40 msgr2=0x7ff06806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.314+0000 7ff08163d640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff06806ae40 0x7ff06806d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff07c0cf700 tx=0x7ff070009040 comp rx=0 tx=0).stop 2023-11-24T06:17:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.314+0000 7ff08163d640 1 -- 172.21.15.5:0/1045766048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07c09cb30 msgr2=0x7ff07c13e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.314+0000 7ff08163d640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07c09cb30 0x7ff07c13e150 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00c420 tx=0x7ff06c00c8e0 comp rx=0 tx=0).stop 2023-11-24T06:17:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.315+0000 7ff08163d640 1 -- 172.21.15.5:0/1045766048 shutdown_connections 2023-11-24T06:17:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.315+0000 7ff08163d640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff06806ae40 0x7ff06806d300 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.315+0000 7ff08163d640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff07c15b350 0x7ff07c1409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.315+0000 7ff08163d640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff07c0ae270 0x7ff07c159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.315+0000 7ff08163d640 1 --2- 172.21.15.5:0/1045766048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff07c09cb30 0x7ff07c13e150 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.315+0000 7ff08163d640 1 -- 172.21.15.5:0/1045766048 >> 172.21.15.5:0/1045766048 conn(0x7ff07c0b0ba0 msgr2=0x7ff07c0aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.315+0000 7ff08163d640 1 -- 172.21.15.5:0/1045766048 shutdown_connections 2023-11-24T06:17:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.315+0000 7ff08163d640 1 -- 172.21.15.5:0/1045766048 wait complete. 2023-11-24T06:17:08.342 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T06:17:08.343 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T06:17:08.343 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:17:08.343 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:17:08.343 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T06:17:08.343 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T06:17:08.344 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:17:08.344 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:17:08.344 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T06:17:08.344 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T06:17:08.344 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:17:08.344 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:17:08.344 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T06:17:08.345 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T06:17:08.345 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:17:08.345 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:17:08.365 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:17:08.365 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T06:17:08.366 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:17:08.366 DEBUG:teuthology.orchestra.run.smithi006:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T06:17:08.402 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:17:08.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T06:17:08.403 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:17:08.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T06:17:08.414 INFO:tasks.ceph.mds.d:Started 2023-11-24T06:17:08.414 DEBUG:teuthology.parallel:result is None 2023-11-24T06:17:08.421 INFO:tasks.ceph.mds.b:Started 2023-11-24T06:17:08.421 DEBUG:teuthology.parallel:result is None 2023-11-24T06:17:08.451 INFO:tasks.ceph.mds.a:Started 2023-11-24T06:17:08.452 DEBUG:teuthology.parallel:result is None 2023-11-24T06:17:08.458 INFO:tasks.ceph.mds.c:Started 2023-11-24T06:17:08.459 DEBUG:teuthology.parallel:result is None 2023-11-24T06:17:08.459 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T06:17:08.578 INFO:tasks.ceph.mds.b.smithi006.stdout:starting mds.b at 2023-11-24T06:17:08.582 INFO:tasks.ceph.mds.d.smithi006.stdout:starting mds.d at 2023-11-24T06:17:08.614 INFO:tasks.ceph.mds.c.smithi005.stdout:starting mds.c at 2023-11-24T06:17:08.615 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2023-11-24T06:17:08.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.651+0000 7f0262cda640 1 Processor -- start 2023-11-24T06:17:08.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.651+0000 7f0262cda640 1 -- start start 2023-11-24T06:17:08.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.652+0000 7f0262cda640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f025c06d520 0x7f025c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.652+0000 7f0262cda640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f025c0b2080 0x7f025c0b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.652+0000 7f0262cda640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c070dc0 con 0x7f025c0acb50 2023-11-24T06:17:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.652+0000 7f0262cda640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c0720e0 con 0x7f025c06d520 2023-11-24T06:17:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.652+0000 7f0262cda640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c161d00 con 0x7f025c0b2080 2023-11-24T06:17:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.652+0000 7f0260a76640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f025c06d520 0x7f025c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.652+0000 7f0260a76640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f025c06d520 0x7f025c06d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53556/0 (socket says 172.21.15.5:53556) 2023-11-24T06:17:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.652+0000 7f0260a76640 1 -- 172.21.15.5:0/3063031847 learned_addr learned my addr 172.21.15.5:0/3063031847 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.653+0000 7f0261277640 1 --2- 172.21.15.5:0/3063031847 >> v2:172.21.15.6:3300/0 conn(0x7f025c0b2080 0x7f025c0b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.653+0000 7f0260a76640 1 -- 172.21.15.5:0/3063031847 >> v2:172.21.15.6:3300/0 conn(0x7f025c0b2080 msgr2=0x7f025c0b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.653+0000 7f0260a76640 1 --2- 172.21.15.5:0/3063031847 >> v2:172.21.15.6:3300/0 conn(0x7f025c0b2080 0x7f025c0b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.653+0000 7f0260a76640 1 -- 172.21.15.5:0/3063031847 >> v1:172.21.15.6:6789/0 conn(0x7f025c0acb50 legacy=0x7f025c0719d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:08.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.653+0000 7f0260a76640 1 -- 172.21.15.5:0/3063031847 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c06de40 con 0x7f025c06d520 2023-11-24T06:17:08.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.653+0000 7f0261277640 1 --2- 172.21.15.5:0/3063031847 >> v2:172.21.15.6:3300/0 conn(0x7f025c0b2080 0x7f025c0b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.654+0000 7f0260a76640 1 --2- 172.21.15.5:0/3063031847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f025c06d520 0x7f025c06d900 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f025c0b19e0 tx=0x7f025002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580c62c24a1f74a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.654+0000 7f025b7fe640 1 -- 172.21.15.5:0/3063031847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250031870 con 0x7f025c06d520 2023-11-24T06:17:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.654+0000 7f025b7fe640 1 -- 172.21.15.5:0/3063031847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02500319d0 con 0x7f025c06d520 2023-11-24T06:17:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.654+0000 7f025b7fe640 1 -- 172.21.15.5:0/3063031847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250031d60 con 0x7f025c06d520 2023-11-24T06:17:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.655+0000 7f0262cda640 1 -- 172.21.15.5:0/3063031847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f025c06d520 msgr2=0x7f025c06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.655+0000 7f0262cda640 1 --2- 172.21.15.5:0/3063031847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f025c06d520 0x7f025c06d900 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f025c0b19e0 tx=0x7f025002b910 comp rx=0 tx=0).stop 2023-11-24T06:17:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.655+0000 7f0262cda640 1 -- 172.21.15.5:0/3063031847 shutdown_connections 2023-11-24T06:17:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.655+0000 7f0262cda640 1 --2- 172.21.15.5:0/3063031847 >> v2:172.21.15.6:3300/0 conn(0x7f025c0b2080 0x7f025c0b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.655+0000 7f0262cda640 1 --2- 172.21.15.5:0/3063031847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f025c06d520 0x7f025c06d900 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.655+0000 7f0262cda640 1 -- 172.21.15.5:0/3063031847 >> 172.21.15.5:0/3063031847 conn(0x7f025c0b0d80 msgr2=0x7f025c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.655+0000 7f0262cda640 1 -- 172.21.15.5:0/3063031847 shutdown_connections 2023-11-24T06:17:08.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.655+0000 7f0262cda640 1 -- 172.21.15.5:0/3063031847 wait complete. 2023-11-24T06:17:08.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.656+0000 7f0262cda640 1 Processor -- start 2023-11-24T06:17:08.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.657+0000 7f0262cda640 1 -- start start 2023-11-24T06:17:08.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.657+0000 7f0262cda640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f025c06d520 0x7f025c0937a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:08.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.657+0000 7f0262cda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f025c0acb50 0x7f025c155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:08.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.658+0000 7f0262cda640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f025c0b2080 0x7f025c0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:08.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.658+0000 7f0262cda640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c094a00 con 0x7f025c0acb50 2023-11-24T06:17:08.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.658+0000 7f0262cda640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c093d80 con 0x7f025c06d520 2023-11-24T06:17:08.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.658+0000 7f0262cda640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c094060 con 0x7f025c0b2080 2023-11-24T06:17:08.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.658+0000 7f025bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f025c0acb50 0x7f025c155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:08.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.658+0000 7f025bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f025c0acb50 0x7f025c155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53562/0 (socket says 172.21.15.5:53562) 2023-11-24T06:17:08.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.658+0000 7f025bfff640 1 -- 172.21.15.5:0/4229568450 learned_addr learned my addr 172.21.15.5:0/4229568450 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:08.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.658+0000 7f0260a76640 1 --2- 172.21.15.5:0/4229568450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f025c06d520 0x7f025c0937a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:08.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.658+0000 7f0261277640 1 --2- 172.21.15.5:0/4229568450 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f025c0b2080 0x7f025c0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:08.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.658+0000 7f025bfff640 1 -- 172.21.15.5:0/4229568450 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f025c0b2080 msgr2=0x7f025c0b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.658+0000 7f025bfff640 1 --2- 172.21.15.5:0/4229568450 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f025c0b2080 0x7f025c0b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.658+0000 7f025bfff640 1 -- 172.21.15.5:0/4229568450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f025c06d520 msgr2=0x7f025c0937a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.658+0000 7f025bfff640 1 --2- 172.21.15.5:0/4229568450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f025c06d520 0x7f025c0937a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.658+0000 7f025bfff640 1 -- 172.21.15.5:0/4229568450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02500094d0 con 0x7f025c0acb50 2023-11-24T06:17:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.659+0000 7f025bfff640 1 --2- 172.21.15.5:0/4229568450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f025c0acb50 0x7f025c155260 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f024800b3f0 tx=0x7f024800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.659+0000 7f02597fa640 1 -- 172.21.15.5:0/4229568450 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248011830 con 0x7f025c0acb50 2023-11-24T06:17:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.659+0000 7f02597fa640 1 -- 172.21.15.5:0/4229568450 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0248011990 con 0x7f025c0acb50 2023-11-24T06:17:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.659+0000 7f02597fa640 1 -- 172.21.15.5:0/4229568450 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248011d40 con 0x7f025c0acb50 2023-11-24T06:17:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.659+0000 7f0262cda640 1 -- 172.21.15.5:0/4229568450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025c094340 con 0x7f025c0acb50 2023-11-24T06:17:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.659+0000 7f0262cda640 1 -- 172.21.15.5:0/4229568450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c1557a0 con 0x7f025c0acb50 2023-11-24T06:17:08.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.660+0000 7f0262cda640 1 -- 172.21.15.5:0/4229568450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c094b40 con 0x7f025c0acb50 2023-11-24T06:17:08.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.661+0000 7f02597fa640 1 -- 172.21.15.5:0/4229568450 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f024801a400 con 0x7f025c0acb50 2023-11-24T06:17:08.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.662+0000 7f02597fa640 1 --2- 172.21.15.5:0/4229568450 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f024006ae40 0x7f024006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:08.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.662+0000 7f02597fa640 1 -- 172.21.15.5:0/4229568450 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0248089320 con 0x7f025c0acb50 2023-11-24T06:17:08.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.663+0000 7f0260a76640 1 --2- 172.21.15.5:0/4229568450 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f024006ae40 0x7f024006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:08.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.663+0000 7f0260a76640 1 --2- 172.21.15.5:0/4229568450 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f024006ae40 0x7f024006d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0250036000 tx=0x7f025003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:08.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.666+0000 7f02597fa640 1 -- 172.21.15.5:0/4229568450 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f025c094b40 con 0x7f025c0acb50 2023-11-24T06:17:08.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:08.873+0000 7f0262cda640 1 -- 172.21.15.5:0/4229568450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f025c094b40 con 0x7f025c0acb50 2023-11-24T06:17:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.346+0000 7f02597fa640 1 -- 172.21.15.5:0/4229568450 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7f025c094b40 con 0x7f025c0acb50 2023-11-24T06:17:09.348 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2023-11-24T06:17:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.349+0000 7f0262cda640 1 -- 172.21.15.5:0/4229568450 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f024006ae40 msgr2=0x7f024006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.349+0000 7f0262cda640 1 --2- 172.21.15.5:0/4229568450 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f024006ae40 0x7f024006d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0250036000 tx=0x7f025003c040 comp rx=0 tx=0).stop 2023-11-24T06:17:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.349+0000 7f0262cda640 1 -- 172.21.15.5:0/4229568450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f025c0acb50 msgr2=0x7f025c155260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.349+0000 7f0262cda640 1 --2- 172.21.15.5:0/4229568450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f025c0acb50 0x7f025c155260 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f024800b3f0 tx=0x7f024800b8b0 comp rx=0 tx=0).stop 2023-11-24T06:17:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.349+0000 7f0262cda640 1 -- 172.21.15.5:0/4229568450 shutdown_connections 2023-11-24T06:17:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.349+0000 7f0262cda640 1 --2- 172.21.15.5:0/4229568450 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f024006ae40 0x7f024006d300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.349+0000 7f0262cda640 1 --2- 172.21.15.5:0/4229568450 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f025c0b2080 0x7f025c0b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.349+0000 7f0262cda640 1 --2- 172.21.15.5:0/4229568450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f025c0acb50 0x7f025c155260 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.349+0000 7f0262cda640 1 --2- 172.21.15.5:0/4229568450 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f025c06d520 0x7f025c0937a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.350+0000 7f0262cda640 1 -- 172.21.15.5:0/4229568450 >> 172.21.15.5:0/4229568450 conn(0x7f025c0b0d80 msgr2=0x7f025c0aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:09.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.350+0000 7f0262cda640 1 -- 172.21.15.5:0/4229568450 shutdown_connections 2023-11-24T06:17:09.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.350+0000 7f0262cda640 1 -- 172.21.15.5:0/4229568450 wait complete. 2023-11-24T06:17:09.379 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T06:17:09.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.568+0000 7f5cca69b640 1 Processor -- start 2023-11-24T06:17:09.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.568+0000 7f5cca69b640 1 -- start start 2023-11-24T06:17:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.570+0000 7f5cca69b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5cc40abff0 0x7f5cc4071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.570+0000 7f5cca69b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc413da30 con 0x7f5cc40b39a0 2023-11-24T06:17:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.570+0000 7f5cca69b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc40ac560 con 0x7f5cc415a100 2023-11-24T06:17:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.570+0000 7f5cca69b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc40ac840 con 0x7f5cc40abff0 2023-11-24T06:17:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.570+0000 7f5cc8c38640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5cc40abff0 0x7f5cc4071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.570+0000 7f5cc37fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5cc40b39a0 0x7f5cc4071130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59142/0 (socket says 172.21.15.5:59142) 2023-11-24T06:17:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.570+0000 7f5cc37fe640 1 -- 172.21.15.5:0/3769948699 learned_addr learned my addr 172.21.15.5:0/3769948699 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.570+0000 7f5cc8c38640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5cc40abff0 0x7f5cc4071840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53568/0 (socket says 172.21.15.5:53568) 2023-11-24T06:17:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.570+0000 7f5cc2ffd640 1 -- 172.21.15.5:0/3769948699 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404949562 0 0) 0x7f5cc413da30 con 0x7f5cc40b39a0 2023-11-24T06:17:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.570+0000 7f5cc8c38640 1 -- 172.21.15.5:0/3769948699 >> v1:172.21.15.6:6789/0 conn(0x7f5cc415a100 legacy=0x7f5cc40b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.571+0000 7f5cc8c38640 1 -- 172.21.15.5:0/3769948699 >> v1:172.21.15.5:6789/0 conn(0x7f5cc40b39a0 legacy=0x7f5cc4071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.571+0000 7f5cc8c38640 1 -- 172.21.15.5:0/3769948699 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc4070650 con 0x7f5cc40abff0 2023-11-24T06:17:09.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.572+0000 7f5cc8c38640 1 --2- 172.21.15.5:0/3769948699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc40abff0 0x7f5cc4071840 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00ba40 tx=0x7f5cbc02e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b7ad1506ad81c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:09.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.572+0000 7f5cc2ffd640 1 -- 172.21.15.5:0/3769948699 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc02ee50 con 0x7f5cc40abff0 2023-11-24T06:17:09.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.572+0000 7f5cc2ffd640 1 -- 172.21.15.5:0/3769948699 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc036440 con 0x7f5cc40abff0 2023-11-24T06:17:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.572+0000 7f5cc2ffd640 1 -- 172.21.15.5:0/3769948699 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0367d0 con 0x7f5cc40abff0 2023-11-24T06:17:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.572+0000 7f5cca69b640 1 -- 172.21.15.5:0/3769948699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc40abff0 msgr2=0x7f5cc4071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.572+0000 7f5cca69b640 1 --2- 172.21.15.5:0/3769948699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc40abff0 0x7f5cc4071840 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00ba40 tx=0x7f5cbc02e0a0 comp rx=0 tx=0).stop 2023-11-24T06:17:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.572+0000 7f5cca69b640 1 -- 172.21.15.5:0/3769948699 shutdown_connections 2023-11-24T06:17:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.572+0000 7f5cca69b640 1 --2- 172.21.15.5:0/3769948699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc40abff0 0x7f5cc4071840 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.573+0000 7f5cca69b640 1 -- 172.21.15.5:0/3769948699 >> 172.21.15.5:0/3769948699 conn(0x7f5cc40b0d80 msgr2=0x7f5cc40b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.573+0000 7f5cca69b640 1 -- 172.21.15.5:0/3769948699 shutdown_connections 2023-11-24T06:17:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.573+0000 7f5cca69b640 1 -- 172.21.15.5:0/3769948699 wait complete. 2023-11-24T06:17:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.574+0000 7f5cca69b640 1 Processor -- start 2023-11-24T06:17:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.574+0000 7f5cca69b640 1 -- start start 2023-11-24T06:17:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.575+0000 7f5cca69b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc40abff0 0x7f5cc415d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.575+0000 7f5cca69b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cc40b39a0 0x7f5cc4090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.575+0000 7f5cca69b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5cc415a100 0x7f5cc40cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.575+0000 7f5cca69b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4091420 con 0x7f5cc40abff0 2023-11-24T06:17:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.575+0000 7f5cca69b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4091700 con 0x7f5cc40b39a0 2023-11-24T06:17:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.575+0000 7f5cca69b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40919e0 con 0x7f5cc415a100 2023-11-24T06:17:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.576+0000 7f5cc3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc40abff0 0x7f5cc415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.576+0000 7f5cc3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc40abff0 0x7f5cc415d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53582/0 (socket says 172.21.15.5:53582) 2023-11-24T06:17:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.576+0000 7f5cc3fff640 1 -- 172.21.15.5:0/642908238 learned_addr learned my addr 172.21.15.5:0/642908238 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.576+0000 7f5cc8c38640 1 --2- 172.21.15.5:0/642908238 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5cc415a100 0x7f5cc40cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.576+0000 7f5cc37fe640 1 --2- 172.21.15.5:0/642908238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cc40b39a0 0x7f5cc4090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.576+0000 7f5cc3fff640 1 -- 172.21.15.5:0/642908238 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5cc415a100 msgr2=0x7f5cc40cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.576+0000 7f5cc3fff640 1 --2- 172.21.15.5:0/642908238 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5cc415a100 0x7f5cc40cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.576+0000 7f5cc3fff640 1 -- 172.21.15.5:0/642908238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cc40b39a0 msgr2=0x7f5cc4090d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.576+0000 7f5cc3fff640 1 --2- 172.21.15.5:0/642908238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cc40b39a0 0x7f5cc4090d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.576+0000 7f5cc3fff640 1 -- 172.21.15.5:0/642908238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc00b6a0 con 0x7f5cc40abff0 2023-11-24T06:17:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.577+0000 7f5cc3fff640 1 --2- 172.21.15.5:0/642908238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc40abff0 0x7f5cc415d230 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400c7c0 tx=0x7f5cb400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.577+0000 7f5cc0ff9640 1 -- 172.21.15.5:0/642908238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb400aa50 con 0x7f5cc40abff0 2023-11-24T06:17:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.577+0000 7f5cc0ff9640 1 -- 172.21.15.5:0/642908238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb400abb0 con 0x7f5cc40abff0 2023-11-24T06:17:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.577+0000 7f5cc0ff9640 1 -- 172.21.15.5:0/642908238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4012420 con 0x7f5cc40abff0 2023-11-24T06:17:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.577+0000 7f5cca69b640 1 -- 172.21.15.5:0/642908238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc4091cc0 con 0x7f5cc40abff0 2023-11-24T06:17:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.577+0000 7f5cca69b640 1 -- 172.21.15.5:0/642908238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc4158de0 con 0x7f5cc40abff0 2023-11-24T06:17:09.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.578+0000 7f5cca69b640 1 -- 172.21.15.5:0/642908238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc4067960 con 0x7f5cc40abff0 2023-11-24T06:17:09.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.579+0000 7f5cc0ff9640 1 -- 172.21.15.5:0/642908238 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5cb4012580 con 0x7f5cc40abff0 2023-11-24T06:17:09.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.580+0000 7f5cc0ff9640 1 --2- 172.21.15.5:0/642908238 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5ca406ae40 0x7f5ca406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.580+0000 7f5cc0ff9640 1 -- 172.21.15.5:0/642908238 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5cb4089490 con 0x7f5cc40abff0 2023-11-24T06:17:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.580+0000 7f5cc37fe640 1 --2- 172.21.15.5:0/642908238 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5ca406ae40 0x7f5ca406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.581+0000 7f5cc37fe640 1 --2- 172.21.15.5:0/642908238 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5ca406ae40 0x7f5ca406d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5cc41f20d0 tx=0x7f5cac007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:09.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.584+0000 7f5cc0ff9640 1 -- 172.21.15.5:0/642908238 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5cb4054920 con 0x7f5cc40abff0 2023-11-24T06:17:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.844+0000 7f5cca69b640 1 -- 172.21.15.5:0/642908238 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5cc4156ad0 con 0x7f5cc40abff0 2023-11-24T06:17:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.845+0000 7f5cc0ff9640 1 -- 172.21.15.5:0/642908238 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f5cb4057f40 con 0x7f5cc40abff0 2023-11-24T06:17:09.847 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:09.848 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-11-24T06:17:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2023-11-24T06:17:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2023-11-24T06:17:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBRP2BlyCrzCBAAJesPTcst5+wrepefjCUvHg==", 2023-11-24T06:17:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2023-11-24T06:17:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2023-11-24T06:17:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2023-11-24T06:17:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2023-11-24T06:17:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBRP2BlVMklLhAAYbNoUFBrX220YLRYI/XfaQ==", 2023-11-24T06:17:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2023-11-24T06:17:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2023-11-24T06:17:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2023-11-24T06:17:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.c", 2023-11-24T06:17:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBRP2BlLkkdGxAAEyqzUDjb03E28fqgPW7Pyw==", 2023-11-24T06:17:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2023-11-24T06:17:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2023-11-24T06:17:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2023-11-24T06:17:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.d", 2023-11-24T06:17:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBSP2Blm77TAxAA0HmGIgVqVEw/cTWRusU5BQ==", 2023-11-24T06:17:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2023-11-24T06:17:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2023-11-24T06:17:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2023-11-24T06:17:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2023-11-24T06:17:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBUP2BlkY6CNxAAO6xMBc0Nn3XUzwMGBQSpCA==", 2023-11-24T06:17:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2023-11-24T06:17:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2023-11-24T06:17:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2023-11-24T06:17:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBYP2BlDfTdFRAA2Uzod+xA1fIi7o5hDEjPlA==", 2023-11-24T06:17:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2023-11-24T06:17:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2023-11-24T06:17:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2023-11-24T06:17:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBbP2BlipkbMRAAWHdPwet86lhQ8vw7+KmMSw==", 2023-11-24T06:17:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2023-11-24T06:17:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2023-11-24T06:17:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2023-11-24T06:17:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBfP2BlcVoWERAA1k6C6OGIqowWtwirtn5plA==", 2023-11-24T06:17:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2023-11-24T06:17:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2023-11-24T06:17:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2023-11-24T06:17:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBmP2BldD2nIRAAWSGnSO7f+h3lMUpR2djc7g==", 2023-11-24T06:17:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2023-11-24T06:17:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2023-11-24T06:17:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2023-11-24T06:17:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBqP2Bll0JtARAAdsNKrm3d7LhvMb8gaLkqFg==", 2023-11-24T06:17:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2023-11-24T06:17:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2023-11-24T06:17:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2023-11-24T06:17:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBtP2Blgr5+DBAARFEkq6Vxl4H3pckjHhnB8A==", 2023-11-24T06:17:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2023-11-24T06:17:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2023-11-24T06:17:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBwP2Bl+VrAJhAAJY4lbiL0VUVC4Psg8at2uQ==", 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBSP2Bl7ShkEhAAJ7REPm4GEzxnc5jqGBmDlQ==", 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2023-11-24T06:17:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2023-11-24T06:17:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.1", 2023-11-24T06:17:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBSP2BlvArOKhAAWiNFDPr+pkxXGvpS22wZwQ==", 2023-11-24T06:17:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2023-11-24T06:17:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2023-11-24T06:17:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2023-11-24T06:17:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2023-11-24T06:17:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.2", 2023-11-24T06:17:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBSP2Bl7UbUHRAAGxHoa/vos5RXgzS1+fswPQ==", 2023-11-24T06:17:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2023-11-24T06:17:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2023-11-24T06:17:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2023-11-24T06:17:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2023-11-24T06:17:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.3", 2023-11-24T06:17:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBSP2Bl/eUYNxAAMHVblVlSugTel+jhnGiEUA==", 2023-11-24T06:17:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2023-11-24T06:17:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2023-11-24T06:17:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2023-11-24T06:17:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2023-11-24T06:17:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2023-11-24T06:17:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBPP2Bl2PD6MBAAkhJdvoHCg6vjTTmSCZal6A==", 2023-11-24T06:17:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2023-11-24T06:17:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2023-11-24T06:17:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2023-11-24T06:17:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-mds", 2023-11-24T06:17:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQB/P2Blp79HAhAA3myREkKhtgYgABTj84S+MQ==", 2023-11-24T06:17:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-mds" 2023-11-24T06:17:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-mgr", 2023-11-24T06:17:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQB/P2BlzeBHAhAAxqshrhSgnbpujvxLWmzY2g==", 2023-11-24T06:17:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-24T06:17:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-osd", 2023-11-24T06:17:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQB/P2BlYPpHAhAAt/0L2VQxhDkBNIEyr1hIVg==", 2023-11-24T06:17:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-osd" 2023-11-24T06:17:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rbd", 2023-11-24T06:17:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQB/P2BlDRRIAhAAPRbmqnHWn6VGUY7byKVQMg==", 2023-11-24T06:17:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-24T06:17:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-24T06:17:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQB/P2Bl2CxIAhAAX03abjJXOX48AkvH5KqX0Q==", 2023-11-24T06:17:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-24T06:17:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rgw", 2023-11-24T06:17:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQB/P2BlEqlIAhAA0aQafGljVNomkXvfFSPNRQ==", 2023-11-24T06:17:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-24T06:17:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBQP2BlG8xzNxAA2ttxOCQMN3FLEsaYKkjPwA==", 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQBQP2BlifcJKRAAtFRCDbxtaYpJsjkEzsiflg==", 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2023-11-24T06:17:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2023-11-24T06:17:09.864 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2023-11-24T06:17:09.864 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2023-11-24T06:17:09.864 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.864 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:09.864 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-11-24T06:17:09.864 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-11-24T06:17:09.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.848+0000 7f5cca69b640 1 -- 172.21.15.5:0/642908238 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5ca406ae40 msgr2=0x7f5ca406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:09.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.848+0000 7f5cca69b640 1 --2- 172.21.15.5:0/642908238 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5ca406ae40 0x7f5ca406d300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5cc41f20d0 tx=0x7f5cac007b50 comp rx=0 tx=0).stop 2023-11-24T06:17:09.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.848+0000 7f5cca69b640 1 -- 172.21.15.5:0/642908238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc40abff0 msgr2=0x7f5cc415d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:09.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.848+0000 7f5cca69b640 1 --2- 172.21.15.5:0/642908238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc40abff0 0x7f5cc415d230 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400c7c0 tx=0x7f5cb400cc80 comp rx=0 tx=0).stop 2023-11-24T06:17:09.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.848+0000 7f5cca69b640 1 -- 172.21.15.5:0/642908238 shutdown_connections 2023-11-24T06:17:09.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.848+0000 7f5cca69b640 1 --2- 172.21.15.5:0/642908238 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5ca406ae40 0x7f5ca406d300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:09.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.848+0000 7f5cca69b640 1 --2- 172.21.15.5:0/642908238 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5cc415a100 0x7f5cc40cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:09.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.848+0000 7f5cca69b640 1 --2- 172.21.15.5:0/642908238 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5cc40b39a0 0x7f5cc4090d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:09.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.848+0000 7f5cca69b640 1 --2- 172.21.15.5:0/642908238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cc40abff0 0x7f5cc415d230 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:09.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.848+0000 7f5cca69b640 1 -- 172.21.15.5:0/642908238 >> 172.21.15.5:0/642908238 conn(0x7f5cc40b0d80 msgr2=0x7f5cc40aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:09.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.848+0000 7f5cca69b640 1 -- 172.21.15.5:0/642908238 shutdown_connections 2023-11-24T06:17:09.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:09.848+0000 7f5cca69b640 1 -- 172.21.15.5:0/642908238 wait complete. 2023-11-24T06:17:09.868 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-24T06:17:10.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.096+0000 7ff384644640 1 Processor -- start 2023-11-24T06:17:10.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.096+0000 7ff384644640 1 -- start start 2023-11-24T06:17:10.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.098+0000 7ff384644640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff37c160970 0x7ff37c0b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:10.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.098+0000 7ff384644640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff37c06dc30 0x7ff37c0acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:10.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.098+0000 7ff384644640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff37c071cd0 con 0x7ff37c0b3140 2023-11-24T06:17:10.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.098+0000 7ff384644640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff37c0b2120 con 0x7ff37c160970 2023-11-24T06:17:10.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.098+0000 7ff384644640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c0b2400 con 0x7ff37c06dc30 2023-11-24T06:17:10.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.098+0000 7ff3823e0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff37c160970 0x7ff37c0b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:10.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.098+0000 7ff3823e0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff37c160970 0x7ff37c0b2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53586/0 (socket says 172.21.15.5:53586) 2023-11-24T06:17:10.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.098+0000 7ff381bdf640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff37c0b3140 0x7ff37c06d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59156/0 (socket says 172.21.15.5:59156) 2023-11-24T06:17:10.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.098+0000 7ff3823e0640 1 -- 172.21.15.5:0/1975700769 learned_addr learned my addr 172.21.15.5:0/1975700769 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:10.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.098+0000 7ff3823e0640 1 -- 172.21.15.5:0/1975700769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff37c06dc30 msgr2=0x7ff37c0acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:10.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.098+0000 7ff382be1640 1 --2- 172.21.15.5:0/1975700769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff37c06dc30 0x7ff37c0acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:10.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.098+0000 7ff3813de640 1 -- 172.21.15.5:0/1975700769 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123220945 0 0) 0x7ff37c071cd0 con 0x7ff37c0b3140 2023-11-24T06:17:10.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.099+0000 7ff3823e0640 1 --2- 172.21.15.5:0/1975700769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff37c06dc30 0x7ff37c0acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.099+0000 7ff3823e0640 1 -- 172.21.15.5:0/1975700769 >> v1:172.21.15.5:6789/0 conn(0x7ff37c0b3140 legacy=0x7ff37c06d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:10.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.099+0000 7ff3823e0640 1 -- 172.21.15.5:0/1975700769 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c0b26e0 con 0x7ff37c160970 2023-11-24T06:17:10.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.099+0000 7ff382be1640 1 --2- 172.21.15.5:0/1975700769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff37c06dc30 0x7ff37c0acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:10.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.100+0000 7ff3823e0640 1 --2- 172.21.15.5:0/1975700769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c160970 0x7ff37c0b2c00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff378009870 tx=0x7ff37802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56a768c4b73a5ed server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.100+0000 7ff3813de640 1 -- 172.21.15.5:0/1975700769 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff378031640 con 0x7ff37c160970 2023-11-24T06:17:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.100+0000 7ff3813de640 1 -- 172.21.15.5:0/1975700769 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3780317a0 con 0x7ff37c160970 2023-11-24T06:17:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.100+0000 7ff3813de640 1 -- 172.21.15.5:0/1975700769 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff378031b30 con 0x7ff37c160970 2023-11-24T06:17:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.101+0000 7ff384644640 1 -- 172.21.15.5:0/1975700769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c160970 msgr2=0x7ff37c0b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.101+0000 7ff384644640 1 --2- 172.21.15.5:0/1975700769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c160970 0x7ff37c0b2c00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff378009870 tx=0x7ff37802b8f0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.101+0000 7ff384644640 1 -- 172.21.15.5:0/1975700769 shutdown_connections 2023-11-24T06:17:10.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.101+0000 7ff384644640 1 --2- 172.21.15.5:0/1975700769 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff37c06dc30 0x7ff37c0acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.101+0000 7ff384644640 1 --2- 172.21.15.5:0/1975700769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c160970 0x7ff37c0b2c00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.101+0000 7ff384644640 1 -- 172.21.15.5:0/1975700769 >> 172.21.15.5:0/1975700769 conn(0x7ff37c0b0d80 msgr2=0x7ff37c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:10.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.101+0000 7ff384644640 1 -- 172.21.15.5:0/1975700769 shutdown_connections 2023-11-24T06:17:10.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.101+0000 7ff384644640 1 -- 172.21.15.5:0/1975700769 wait complete. 2023-11-24T06:17:10.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.103+0000 7ff384644640 1 Processor -- start 2023-11-24T06:17:10.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.103+0000 7ff384644640 1 -- start start 2023-11-24T06:17:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.104+0000 7ff384644640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff37c06dc30 0x7ff37c155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.104+0000 7ff384644640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c0b3140 0x7ff37c09b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.104+0000 7ff384644640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff37c160970 0x7ff37c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.104+0000 7ff384644640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c09bb90 con 0x7ff37c0b3140 2023-11-24T06:17:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.104+0000 7ff384644640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c09be70 con 0x7ff37c06dc30 2023-11-24T06:17:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.104+0000 7ff384644640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c09c150 con 0x7ff37c160970 2023-11-24T06:17:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.104+0000 7ff381bdf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c0b3140 0x7ff37c09b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.104+0000 7ff381bdf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c0b3140 0x7ff37c09b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53590/0 (socket says 172.21.15.5:53590) 2023-11-24T06:17:10.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.104+0000 7ff381bdf640 1 -- 172.21.15.5:0/1261835589 learned_addr learned my addr 172.21.15.5:0/1261835589 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:10.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.104+0000 7ff381bdf640 1 -- 172.21.15.5:0/1261835589 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff37c160970 msgr2=0x7ff37c06c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:10.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.104+0000 7ff381bdf640 1 --2- 172.21.15.5:0/1261835589 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff37c160970 0x7ff37c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.104+0000 7ff381bdf640 1 -- 172.21.15.5:0/1261835589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff37c06dc30 msgr2=0x7ff37c155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:10.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.104+0000 7ff381bdf640 1 --2- 172.21.15.5:0/1261835589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff37c06dc30 0x7ff37c155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.104+0000 7ff381bdf640 1 -- 172.21.15.5:0/1261835589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3780094d0 con 0x7ff37c0b3140 2023-11-24T06:17:10.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.105+0000 7ff381bdf640 1 --2- 172.21.15.5:0/1261835589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c0b3140 0x7ff37c09b340 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c4d0 tx=0x7ff36c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:10.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.105+0000 7ff372ffd640 1 -- 172.21.15.5:0/1261835589 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c0074e0 con 0x7ff37c0b3140 2023-11-24T06:17:10.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.105+0000 7ff372ffd640 1 -- 172.21.15.5:0/1261835589 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c007660 con 0x7ff37c0b3140 2023-11-24T06:17:10.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.105+0000 7ff372ffd640 1 -- 172.21.15.5:0/1261835589 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c004a20 con 0x7ff37c0b3140 2023-11-24T06:17:10.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.105+0000 7ff384644640 1 -- 172.21.15.5:0/1261835589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff37c09c430 con 0x7ff37c0b3140 2023-11-24T06:17:10.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.105+0000 7ff384644640 1 -- 172.21.15.5:0/1261835589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37c09b880 con 0x7ff37c0b3140 2023-11-24T06:17:10.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.106+0000 7ff384644640 1 -- 172.21.15.5:0/1261835589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff37c15a6f0 con 0x7ff37c0b3140 2023-11-24T06:17:10.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.107+0000 7ff372ffd640 1 -- 172.21.15.5:0/1261835589 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff36c004b80 con 0x7ff37c0b3140 2023-11-24T06:17:10.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.108+0000 7ff372ffd640 1 --2- 172.21.15.5:0/1261835589 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff36006ae40 0x7ff36006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:10.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.108+0000 7ff372ffd640 1 -- 172.21.15.5:0/1261835589 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff36c088550 con 0x7ff37c0b3140 2023-11-24T06:17:10.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.109+0000 7ff3823e0640 1 --2- 172.21.15.5:0/1261835589 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff36006ae40 0x7ff36006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:10.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.109+0000 7ff3823e0640 1 --2- 172.21.15.5:0/1261835589 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff36006ae40 0x7ff36006d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff378036000 tx=0x7ff378030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.116+0000 7ff372ffd640 1 -- 172.21.15.5:0/1261835589 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff36c0538b0 con 0x7ff37c0b3140 2023-11-24T06:17:10.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.391+0000 7ff384644640 1 -- 172.21.15.5:0/1261835589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7ff37c074770 con 0x7ff37c0b3140 2023-11-24T06:17:10.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.395+0000 7ff372ffd640 1 -- 172.21.15.5:0/1261835589 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff36c0180a0 con 0x7ff37c0b3140 2023-11-24T06:17:10.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.397+0000 7ff384644640 1 -- 172.21.15.5:0/1261835589 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff36006ae40 msgr2=0x7ff36006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:10.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.397+0000 7ff384644640 1 --2- 172.21.15.5:0/1261835589 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff36006ae40 0x7ff36006d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff378036000 tx=0x7ff378030000 comp rx=0 tx=0).stop 2023-11-24T06:17:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.397+0000 7ff384644640 1 -- 172.21.15.5:0/1261835589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c0b3140 msgr2=0x7ff37c09b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.397+0000 7ff384644640 1 --2- 172.21.15.5:0/1261835589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c0b3140 0x7ff37c09b340 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c4d0 tx=0x7ff36c00c990 comp rx=0 tx=0).stop 2023-11-24T06:17:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.397+0000 7ff384644640 1 -- 172.21.15.5:0/1261835589 shutdown_connections 2023-11-24T06:17:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.397+0000 7ff384644640 1 --2- 172.21.15.5:0/1261835589 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff36006ae40 0x7ff36006d300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.397+0000 7ff384644640 1 --2- 172.21.15.5:0/1261835589 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff37c160970 0x7ff37c06c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.397+0000 7ff384644640 1 --2- 172.21.15.5:0/1261835589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c0b3140 0x7ff37c09b340 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.397+0000 7ff384644640 1 --2- 172.21.15.5:0/1261835589 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff37c06dc30 0x7ff37c155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.397+0000 7ff384644640 1 -- 172.21.15.5:0/1261835589 >> 172.21.15.5:0/1261835589 conn(0x7ff37c0b0d80 msgr2=0x7ff37c1619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.398+0000 7ff384644640 1 -- 172.21.15.5:0/1261835589 shutdown_connections 2023-11-24T06:17:10.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.398+0000 7ff384644640 1 -- 172.21.15.5:0/1261835589 wait complete. 2023-11-24T06:17:10.418 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-24T06:17:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.623+0000 7f5f0ffff640 1 Processor -- start 2023-11-24T06:17:10.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.624+0000 7f5f0ffff640 1 -- start start 2023-11-24T06:17:10.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.625+0000 7f5f0ffff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5f1015ed20 0x7f5f10141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:10.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.625+0000 7f5f0ffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f10071820 0x7f5f1006cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:10.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.625+0000 7f5f0ffff640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1006cae0 con 0x7f5f1015f100 2023-11-24T06:17:10.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.625+0000 7f5f0ffff640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f10071e40 con 0x7f5f1015ed20 2023-11-24T06:17:10.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.625+0000 7f5f0ffff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1006c1b0 con 0x7f5f10071820 2023-11-24T06:17:10.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.626+0000 7f5f0effd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5f1015ed20 0x7f5f10141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:10.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.626+0000 7f5f0f7fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f10071820 0x7f5f1006cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:10.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.626+0000 7f5f0effd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5f1015ed20 0x7f5f10141e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:49966/0 (socket says 172.21.15.5:49966) 2023-11-24T06:17:10.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.626+0000 7f5f0effd640 1 -- 172.21.15.5:0/397368785 learned_addr learned my addr 172.21.15.5:0/397368785 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.627+0000 7f5f0effd640 1 -- 172.21.15.5:0/397368785 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f10071820 msgr2=0x7f5f1006cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.627+0000 7f5f0effd640 1 --2- 172.21.15.5:0/397368785 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f10071820 0x7f5f1006cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.627+0000 7f5f0effd640 1 -- 172.21.15.5:0/397368785 >> v1:172.21.15.6:6789/0 conn(0x7f5f1015f100 legacy=0x7f5f10071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:10.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.627+0000 7f5f0effd640 1 -- 172.21.15.5:0/397368785 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1006c490 con 0x7f5f1015ed20 2023-11-24T06:17:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.628+0000 7f5f0effd640 1 --2- 172.21.15.5:0/397368785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f1015ed20 0x7f5f10141e80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009870 tx=0x7f5f0002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=551d0e7ff8d30ee7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.629+0000 7f5f0e7fc640 1 -- 172.21.15.5:0/397368785 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00030840 con 0x7f5f1015ed20 2023-11-24T06:17:10.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.629+0000 7f5f0e7fc640 1 -- 172.21.15.5:0/397368785 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f000309a0 con 0x7f5f1015ed20 2023-11-24T06:17:10.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.629+0000 7f5f0e7fc640 1 -- 172.21.15.5:0/397368785 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f00030d30 con 0x7f5f1015ed20 2023-11-24T06:17:10.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.629+0000 7f5f0ffff640 1 -- 172.21.15.5:0/397368785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f1015ed20 msgr2=0x7f5f10141e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:10.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.629+0000 7f5f0ffff640 1 --2- 172.21.15.5:0/397368785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f1015ed20 0x7f5f10141e80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5f00009870 tx=0x7f5f0002b980 comp rx=0 tx=0).stop 2023-11-24T06:17:10.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.629+0000 7f5f0ffff640 1 -- 172.21.15.5:0/397368785 shutdown_connections 2023-11-24T06:17:10.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.629+0000 7f5f0ffff640 1 --2- 172.21.15.5:0/397368785 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f10071820 0x7f5f1006cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.629+0000 7f5f0ffff640 1 --2- 172.21.15.5:0/397368785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f1015ed20 0x7f5f10141e80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.629+0000 7f5f0ffff640 1 -- 172.21.15.5:0/397368785 >> 172.21.15.5:0/397368785 conn(0x7f5f100b0840 msgr2=0x7f5f100b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.629+0000 7f5f0ffff640 1 -- 172.21.15.5:0/397368785 shutdown_connections 2023-11-24T06:17:10.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.629+0000 7f5f0ffff640 1 -- 172.21.15.5:0/397368785 wait complete. 2023-11-24T06:17:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.631+0000 7f5f0ffff640 1 Processor -- start 2023-11-24T06:17:10.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.631+0000 7f5f0ffff640 1 -- start start 2023-11-24T06:17:10.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.632+0000 7f5f0ffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f10071820 0x7f5f1013e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:10.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.632+0000 7f5f0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1015ed20 0x7f5f10150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.632+0000 7f5f0ffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f1015f100 0x7f5f10139d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.632+0000 7f5f0ffff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f10150a80 con 0x7f5f1015ed20 2023-11-24T06:17:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.632+0000 7f5f0ffff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1013a2d0 con 0x7f5f10071820 2023-11-24T06:17:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.632+0000 7f5f0ffff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1013a520 con 0x7f5f1015f100 2023-11-24T06:17:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.633+0000 7f5f07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1015ed20 0x7f5f10150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.633+0000 7f5f07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1015ed20 0x7f5f10150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53604/0 (socket says 172.21.15.5:53604) 2023-11-24T06:17:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.633+0000 7f5f07fff640 1 -- 172.21.15.5:0/4289179193 learned_addr learned my addr 172.21.15.5:0/4289179193 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.633+0000 7f5f0effd640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f10071820 0x7f5f1013e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:10.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.633+0000 7f5f0f7fe640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f1015f100 0x7f5f10139d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.633+0000 7f5f07fff640 1 -- 172.21.15.5:0/4289179193 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f1015f100 msgr2=0x7f5f10139d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.633+0000 7f5f07fff640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f1015f100 0x7f5f10139d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.633+0000 7f5f07fff640 1 -- 172.21.15.5:0/4289179193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f10071820 msgr2=0x7f5f1013e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.633+0000 7f5f07fff640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f10071820 0x7f5f1013e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.633+0000 7f5f07fff640 1 -- 172.21.15.5:0/4289179193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f000094d0 con 0x7f5f1015ed20 2023-11-24T06:17:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.633+0000 7f5f0f7fe640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f1015f100 0x7f5f10139d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:17:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.634+0000 7f5f07fff640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1015ed20 0x7f5f10150540 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800c3c0 tx=0x7f5ef800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:10.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.634+0000 7f5f077fe640 1 -- 172.21.15.5:0/4289179193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef80052d0 con 0x7f5f1015ed20 2023-11-24T06:17:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.634+0000 7f5f077fe640 1 -- 172.21.15.5:0/4289179193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef8005450 con 0x7f5f1015ed20 2023-11-24T06:17:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.634+0000 7f5f077fe640 1 -- 172.21.15.5:0/4289179193 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8007260 con 0x7f5f1015ed20 2023-11-24T06:17:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.634+0000 7f5f0ffff640 1 -- 172.21.15.5:0/4289179193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1013a800 con 0x7f5f1015ed20 2023-11-24T06:17:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.634+0000 7f5f0ffff640 1 -- 172.21.15.5:0/4289179193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1013adc0 con 0x7f5f1015ed20 2023-11-24T06:17:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.635+0000 7f5f0ffff640 1 -- 172.21.15.5:0/4289179193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f10067d10 con 0x7f5f1015ed20 2023-11-24T06:17:10.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.636+0000 7f5f077fe640 1 -- 172.21.15.5:0/4289179193 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ef80073c0 con 0x7f5f1015ed20 2023-11-24T06:17:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.638+0000 7f5f077fe640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5ef006ae40 0x7f5ef006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.638+0000 7f5f077fe640 1 -- 172.21.15.5:0/4289179193 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5ef8088650 con 0x7f5f1015ed20 2023-11-24T06:17:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.638+0000 7f5f0effd640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5ef006ae40 0x7f5ef006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:10.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.639+0000 7f5f0effd640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5ef006ae40 0x7f5ef006d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f00036000 tx=0x7f5f00008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:10.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.642+0000 7f5f077fe640 1 -- 172.21.15.5:0/4289179193 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ef8053a60 con 0x7f5f1015ed20 2023-11-24T06:17:10.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.893+0000 7f5f0ffff640 1 -- 172.21.15.5:0/4289179193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f5f1004e840 con 0x7f5f1015ed20 2023-11-24T06:17:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.897+0000 7f5f077fe640 1 -- 172.21.15.5:0/4289179193 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5ef8057080 con 0x7f5f1015ed20 2023-11-24T06:17:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.900+0000 7f5f0ffff640 1 -- 172.21.15.5:0/4289179193 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5ef006ae40 msgr2=0x7f5ef006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.900+0000 7f5f0ffff640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5ef006ae40 0x7f5ef006d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5f00036000 tx=0x7f5f00008030 comp rx=0 tx=0).stop 2023-11-24T06:17:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.900+0000 7f5f0ffff640 1 -- 172.21.15.5:0/4289179193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1015ed20 msgr2=0x7f5f10150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.900+0000 7f5f0ffff640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1015ed20 0x7f5f10150540 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800c3c0 tx=0x7f5ef800c880 comp rx=0 tx=0).stop 2023-11-24T06:17:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.900+0000 7f5f0ffff640 1 -- 172.21.15.5:0/4289179193 shutdown_connections 2023-11-24T06:17:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.900+0000 7f5f0ffff640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5ef006ae40 0x7f5ef006d300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.900+0000 7f5f0ffff640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5f1015f100 0x7f5f10139d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.900+0000 7f5f0ffff640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f1015ed20 0x7f5f10150540 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.900+0000 7f5f0ffff640 1 --2- 172.21.15.5:0/4289179193 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5f10071820 0x7f5f1013e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.900+0000 7f5f0ffff640 1 -- 172.21.15.5:0/4289179193 >> 172.21.15.5:0/4289179193 conn(0x7f5f100b0840 msgr2=0x7f5f100afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.901+0000 7f5f0ffff640 1 -- 172.21.15.5:0/4289179193 shutdown_connections 2023-11-24T06:17:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:10.901+0000 7f5f0ffff640 1 -- 172.21.15.5:0/4289179193 wait complete. 2023-11-24T06:17:10.921 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-24T06:17:11.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.138+0000 7f945214e640 1 Processor -- start 2023-11-24T06:17:11.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.138+0000 7f945214e640 1 -- start start 2023-11-24T06:17:11.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.139+0000 7f945214e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f944c143b30 0x7f944c06c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:11.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.139+0000 7f945214e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f944c15e3d0 0x7f944c0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:11.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.139+0000 7f945214e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f944c140bc0 con 0x7f944c1407e0 2023-11-24T06:17:11.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.139+0000 7f945214e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944c0b1be0 con 0x7f944c143b30 2023-11-24T06:17:11.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.139+0000 7f945214e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944c0b1ec0 con 0x7f944c15e3d0 2023-11-24T06:17:11.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.140+0000 7f945094b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f944c143b30 0x7f944c06c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:11.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.140+0000 7f945194d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f944c15e3d0 0x7f944c0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:11.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.140+0000 7f945094b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f944c143b30 0x7f944c06c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:49986/0 (socket says 172.21.15.5:49986) 2023-11-24T06:17:11.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.140+0000 7f945094b640 1 -- 172.21.15.5:0/4156265415 learned_addr learned my addr 172.21.15.5:0/4156265415 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:11.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.141+0000 7f945094b640 1 -- 172.21.15.5:0/4156265415 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f944c15e3d0 msgr2=0x7f944c0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:11.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.141+0000 7f945094b640 1 --2- 172.21.15.5:0/4156265415 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f944c15e3d0 0x7f944c0b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.141+0000 7f945094b640 1 -- 172.21.15.5:0/4156265415 >> v1:172.21.15.6:6789/0 conn(0x7f944c1407e0 legacy=0x7f944c15dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:11.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.141+0000 7f945094b640 1 -- 172.21.15.5:0/4156265415 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944c0b21a0 con 0x7f944c143b30 2023-11-24T06:17:11.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.142+0000 7f945094b640 1 --2- 172.21.15.5:0/4156265415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f944c143b30 0x7f944c06c320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f943c00bb10 tx=0x7f943c02e150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc7bc564d1cd9351 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:11.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.143+0000 7f9443fff640 1 -- 172.21.15.5:0/4156265415 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c036440 con 0x7f944c143b30 2023-11-24T06:17:11.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.143+0000 7f9443fff640 1 -- 172.21.15.5:0/4156265415 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c0365a0 con 0x7f944c143b30 2023-11-24T06:17:11.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.143+0000 7f9443fff640 1 -- 172.21.15.5:0/4156265415 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c036930 con 0x7f944c143b30 2023-11-24T06:17:11.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.144+0000 7f945214e640 1 -- 172.21.15.5:0/4156265415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f944c143b30 msgr2=0x7f944c06c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:11.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.144+0000 7f945214e640 1 --2- 172.21.15.5:0/4156265415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f944c143b30 0x7f944c06c320 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f943c00bb10 tx=0x7f943c02e150 comp rx=0 tx=0).stop 2023-11-24T06:17:11.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.144+0000 7f945214e640 1 -- 172.21.15.5:0/4156265415 shutdown_connections 2023-11-24T06:17:11.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.144+0000 7f945214e640 1 --2- 172.21.15.5:0/4156265415 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f944c15e3d0 0x7f944c0b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.144+0000 7f945214e640 1 --2- 172.21.15.5:0/4156265415 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f944c143b30 0x7f944c06c320 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.144+0000 7f945214e640 1 -- 172.21.15.5:0/4156265415 >> 172.21.15.5:0/4156265415 conn(0x7f944c0b0840 msgr2=0x7f944c0b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:11.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.144+0000 7f945214e640 1 -- 172.21.15.5:0/4156265415 shutdown_connections 2023-11-24T06:17:11.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.144+0000 7f945214e640 1 -- 172.21.15.5:0/4156265415 wait complete. 2023-11-24T06:17:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.146+0000 7f945214e640 1 Processor -- start 2023-11-24T06:17:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.146+0000 7f945214e640 1 -- start start 2023-11-24T06:17:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.147+0000 7f945214e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f944c1407e0 0x7f944c0cf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.147+0000 7f945214e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f944c143b30 0x7f944c13d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:11.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.147+0000 7f945214e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f944c15e3d0 0x7f944c0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.147+0000 7f945214e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c0ce5c0 con 0x7f944c143b30 2023-11-24T06:17:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.147+0000 7f945214e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944c0ce8a0 con 0x7f944c1407e0 2023-11-24T06:17:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.147+0000 7f945214e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944c0ceb80 con 0x7f944c15e3d0 2023-11-24T06:17:11.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.147+0000 7f945094b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f944c143b30 0x7f944c13d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.147+0000 7f945094b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f944c143b30 0x7f944c13d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53614/0 (socket says 172.21.15.5:53614) 2023-11-24T06:17:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.147+0000 7f945094b640 1 -- 172.21.15.5:0/517228794 learned_addr learned my addr 172.21.15.5:0/517228794 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.147+0000 7f945114c640 1 --2- 172.21.15.5:0/517228794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f944c1407e0 0x7f944c0cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.147+0000 7f945194d640 1 --2- 172.21.15.5:0/517228794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f944c15e3d0 0x7f944c0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.148+0000 7f945094b640 1 -- 172.21.15.5:0/517228794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f944c15e3d0 msgr2=0x7f944c0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.148+0000 7f945094b640 1 --2- 172.21.15.5:0/517228794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f944c15e3d0 0x7f944c0cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.148+0000 7f945094b640 1 -- 172.21.15.5:0/517228794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f944c1407e0 msgr2=0x7f944c0cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:11.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.148+0000 7f945094b640 1 --2- 172.21.15.5:0/517228794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f944c1407e0 0x7f944c0cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.148+0000 7f945094b640 1 -- 172.21.15.5:0/517228794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c00b770 con 0x7f944c143b30 2023-11-24T06:17:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.148+0000 7f945094b640 1 --2- 172.21.15.5:0/517228794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f944c143b30 0x7f944c13d590 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f943c00b2c0 tx=0x7f943c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.148+0000 7f9441ffb640 1 -- 172.21.15.5:0/517228794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c036440 con 0x7f944c143b30 2023-11-24T06:17:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.148+0000 7f9441ffb640 1 -- 172.21.15.5:0/517228794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c034980 con 0x7f944c143b30 2023-11-24T06:17:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.148+0000 7f9441ffb640 1 -- 172.21.15.5:0/517228794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c034d50 con 0x7f944c143b30 2023-11-24T06:17:11.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.148+0000 7f945214e640 1 -- 172.21.15.5:0/517228794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f944c0cee60 con 0x7f944c143b30 2023-11-24T06:17:11.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.148+0000 7f945214e640 1 -- 172.21.15.5:0/517228794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944c13dad0 con 0x7f944c143b30 2023-11-24T06:17:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.150+0000 7f945214e640 1 -- 172.21.15.5:0/517228794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f944c0cf380 con 0x7f944c143b30 2023-11-24T06:17:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.150+0000 7f9441ffb640 1 -- 172.21.15.5:0/517228794 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f943c040400 con 0x7f944c143b30 2023-11-24T06:17:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.151+0000 7f9441ffb640 1 --2- 172.21.15.5:0/517228794 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f943006ae40 0x7f943006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:11.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.151+0000 7f9441ffb640 1 -- 172.21.15.5:0/517228794 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f943c0ae110 con 0x7f944c143b30 2023-11-24T06:17:11.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.152+0000 7f945114c640 1 --2- 172.21.15.5:0/517228794 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f943006ae40 0x7f943006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:11.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.152+0000 7f945114c640 1 --2- 172.21.15.5:0/517228794 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f943006ae40 0x7f943006d300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f944c0cd9f0 tx=0x7f9448009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:11.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.156+0000 7f9441ffb640 1 -- 172.21.15.5:0/517228794 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f944c0cf380 con 0x7f944c143b30 2023-11-24T06:17:11.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.408+0000 7f945214e640 1 -- 172.21.15.5:0/517228794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f944c0cf380 con 0x7f944c143b30 2023-11-24T06:17:11.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.413+0000 7f9441ffb640 1 -- 172.21.15.5:0/517228794 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f944c0cf380 con 0x7f944c143b30 2023-11-24T06:17:11.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.416+0000 7f945214e640 1 -- 172.21.15.5:0/517228794 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f943006ae40 msgr2=0x7f943006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:11.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.416+0000 7f945214e640 1 --2- 172.21.15.5:0/517228794 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f943006ae40 0x7f943006d300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f944c0cd9f0 tx=0x7f9448009040 comp rx=0 tx=0).stop 2023-11-24T06:17:11.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.416+0000 7f945214e640 1 -- 172.21.15.5:0/517228794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f944c143b30 msgr2=0x7f944c13d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:11.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.416+0000 7f945214e640 1 --2- 172.21.15.5:0/517228794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f944c143b30 0x7f944c13d590 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f943c00b2c0 tx=0x7f943c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.416+0000 7f945214e640 1 -- 172.21.15.5:0/517228794 shutdown_connections 2023-11-24T06:17:11.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.416+0000 7f945214e640 1 --2- 172.21.15.5:0/517228794 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f943006ae40 0x7f943006d300 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.416+0000 7f945214e640 1 --2- 172.21.15.5:0/517228794 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f944c15e3d0 0x7f944c0cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.416+0000 7f945214e640 1 --2- 172.21.15.5:0/517228794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f944c143b30 0x7f944c13d590 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.416+0000 7f945214e640 1 --2- 172.21.15.5:0/517228794 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f944c1407e0 0x7f944c0cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.416+0000 7f945214e640 1 -- 172.21.15.5:0/517228794 >> 172.21.15.5:0/517228794 conn(0x7f944c0b0840 msgr2=0x7f944c0b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:11.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.416+0000 7f945214e640 1 -- 172.21.15.5:0/517228794 shutdown_connections 2023-11-24T06:17:11.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.416+0000 7f945214e640 1 -- 172.21.15.5:0/517228794 wait complete. 2023-11-24T06:17:11.435 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-24T06:17:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.641+0000 7f306517c640 1 Processor -- start 2023-11-24T06:17:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.641+0000 7f306517c640 1 -- start start 2023-11-24T06:17:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.642+0000 7f306517c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30600b39a0 0x7f3060071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.642+0000 7f306517c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306013da30 con 0x7f306015a100 2023-11-24T06:17:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.642+0000 7f306517c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30600ac560 con 0x7f30600abff0 2023-11-24T06:17:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.642+0000 7f306517c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30600ac840 con 0x7f30600b39a0 2023-11-24T06:17:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.642+0000 7f305ed9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f306015a100 0x7f30600b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59166/0 (socket says 172.21.15.5:59166) 2023-11-24T06:17:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.642+0000 7f305ed9d640 1 -- 172.21.15.5:0/2198123628 learned_addr learned my addr 172.21.15.5:0/2198123628 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.643+0000 7f305dd9b640 1 -- 172.21.15.5:0/2198123628 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636525800 0 0) 0x7f306013da30 con 0x7f306015a100 2023-11-24T06:17:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.643+0000 7f305dd9b640 1 -- 172.21.15.5:0/2198123628 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f304c005730 con 0x7f306015a100 2023-11-24T06:17:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.643+0000 7f305e59c640 1 --2- 172.21.15.5:0/2198123628 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30600b39a0 0x7f3060071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.643+0000 7f305dd9b640 1 -- 172.21.15.5:0/2198123628 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2754627427 0 0) 0x7f304c005730 con 0x7f306015a100 2023-11-24T06:17:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.643+0000 7f305dd9b640 1 -- 172.21.15.5:0/2198123628 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30600b39a0 msgr2=0x7f3060071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.643+0000 7f305dd9b640 1 --2- 172.21.15.5:0/2198123628 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30600b39a0 0x7f3060071130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.643+0000 7f305dd9b640 1 -- 172.21.15.5:0/2198123628 >> v1:172.21.15.6:6789/0 conn(0x7f30600abff0 legacy=0x7f3060071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.643+0000 7f305dd9b640 1 -- 172.21.15.5:0/2198123628 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3060070650 con 0x7f306015a100 2023-11-24T06:17:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.643+0000 7f305dd9b640 1 -- 172.21.15.5:0/2198123628 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f30500019f0 con 0x7f306015a100 2023-11-24T06:17:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.643+0000 7f305dd9b640 1 -- 172.21.15.5:0/2198123628 >> v1:172.21.15.5:6789/0 conn(0x7f306015a100 legacy=0x7f30600b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.643+0000 7f305dd9b640 1 --2- 172.21.15.5:0/2198123628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f304c005bb0 0x7f304c005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.643+0000 7f305dd9b640 1 --2- 172.21.15.5:0/2198123628 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f304c0069d0 0x7f304c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.644+0000 7f305dd9b640 1 --2- 172.21.15.5:0/2198123628 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30600b39a0 0x7f304c007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.644+0000 7f305dd9b640 1 -- 172.21.15.5:0/2198123628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30600ac840 con 0x7f304c005bb0 2023-11-24T06:17:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.644+0000 7f305dd9b640 1 -- 172.21.15.5:0/2198123628 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c00dd20 con 0x7f304c0069d0 2023-11-24T06:17:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.644+0000 7f305dd9b640 1 -- 172.21.15.5:0/2198123628 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c00dee0 con 0x7f30600b39a0 2023-11-24T06:17:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.644+0000 7f305ed9d640 1 --2- 172.21.15.5:0/2198123628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f304c005bb0 0x7f304c005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.644+0000 7f305e59c640 1 --2- 172.21.15.5:0/2198123628 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f304c0069d0 0x7f304c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.644+0000 7f305f59e640 1 --2- 172.21.15.5:0/2198123628 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30600b39a0 0x7f304c007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.644+0000 7f305ed9d640 1 -- 172.21.15.5:0/2198123628 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30600b39a0 msgr2=0x7f304c007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.644+0000 7f305ed9d640 1 --2- 172.21.15.5:0/2198123628 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30600b39a0 0x7f304c007420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.644+0000 7f305ed9d640 1 -- 172.21.15.5:0/2198123628 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f304c0069d0 msgr2=0x7f304c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.644+0000 7f305ed9d640 1 --2- 172.21.15.5:0/2198123628 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f304c0069d0 0x7f304c006e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.644+0000 7f305ed9d640 1 -- 172.21.15.5:0/2198123628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c005730 con 0x7f304c005bb0 2023-11-24T06:17:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.645+0000 7f305ed9d640 1 --2- 172.21.15.5:0/2198123628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f304c005bb0 0x7f304c005fd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f305000a760 tx=0x7f304c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889fa0d92d77ad0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.646+0000 7f305dd9b640 1 -- 172.21.15.5:0/2198123628 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050034630 con 0x7f304c005bb0 2023-11-24T06:17:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.646+0000 7f305dd9b640 1 -- 172.21.15.5:0/2198123628 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305002f070 con 0x7f304c005bb0 2023-11-24T06:17:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.646+0000 7f305dd9b640 1 -- 172.21.15.5:0/2198123628 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30500349f0 con 0x7f304c005bb0 2023-11-24T06:17:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.646+0000 7f306517c640 1 -- 172.21.15.5:0/2198123628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f304c005bb0 msgr2=0x7f304c005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:11.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.646+0000 7f306517c640 1 --2- 172.21.15.5:0/2198123628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f304c005bb0 0x7f304c005fd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f305000a760 tx=0x7f304c007a00 comp rx=0 tx=0).stop 2023-11-24T06:17:11.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.646+0000 7f305f59e640 1 -- 172.21.15.5:0/2198123628 reap_dead start 2023-11-24T06:17:11.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.646+0000 7f306517c640 1 -- 172.21.15.5:0/2198123628 shutdown_connections 2023-11-24T06:17:11.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.646+0000 7f306517c640 1 -- 172.21.15.5:0/2198123628 >> 172.21.15.5:0/2198123628 conn(0x7f30600b0d80 msgr2=0x7f30600b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:11.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.646+0000 7f306517c640 1 -- 172.21.15.5:0/2198123628 shutdown_connections 2023-11-24T06:17:11.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.646+0000 7f306517c640 1 -- 172.21.15.5:0/2198123628 wait complete. 2023-11-24T06:17:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.648+0000 7f306517c640 1 Processor -- start 2023-11-24T06:17:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.648+0000 7f306517c640 1 -- start start 2023-11-24T06:17:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.649+0000 7f306517c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f304c005bb0 0x7f30601f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.649+0000 7f306517c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30600ca6f0 0x7f30600ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.649+0000 7f306517c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30600cab70 0x7f30600cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.649+0000 7f306517c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30600c9de0 con 0x7f304c005bb0 2023-11-24T06:17:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.649+0000 7f306517c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30600c91d0 con 0x7f30600ca6f0 2023-11-24T06:17:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.649+0000 7f306517c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30600c94b0 con 0x7f30600cab70 2023-11-24T06:17:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.649+0000 7f305ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f304c005bb0 0x7f30601f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.649+0000 7f305ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f304c005bb0 0x7f30601f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53632/0 (socket says 172.21.15.5:53632) 2023-11-24T06:17:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.649+0000 7f305ed9d640 1 -- 172.21.15.5:0/689843518 learned_addr learned my addr 172.21.15.5:0/689843518 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.650+0000 7f305ed9d640 1 -- 172.21.15.5:0/689843518 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30600cab70 msgr2=0x7f30600cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.650+0000 7f305ed9d640 1 --2- 172.21.15.5:0/689843518 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30600cab70 0x7f30600cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.650+0000 7f305ed9d640 1 -- 172.21.15.5:0/689843518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30600ca6f0 msgr2=0x7f30600ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.650+0000 7f305ed9d640 1 --2- 172.21.15.5:0/689843518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30600ca6f0 0x7f30600ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.650+0000 7f305ed9d640 1 -- 172.21.15.5:0/689843518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30600c9790 con 0x7f304c005bb0 2023-11-24T06:17:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.650+0000 7f305ed9d640 1 --2- 172.21.15.5:0/689843518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f304c005bb0 0x7f30601f2fb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3050000f80 tx=0x7f3050001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.650+0000 7f30577fe640 1 -- 172.21.15.5:0/689843518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050034900 con 0x7f304c005bb0 2023-11-24T06:17:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.650+0000 7f30577fe640 1 -- 172.21.15.5:0/689843518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3050034a60 con 0x7f304c005bb0 2023-11-24T06:17:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.650+0000 7f30577fe640 1 -- 172.21.15.5:0/689843518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050034df0 con 0x7f304c005bb0 2023-11-24T06:17:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.650+0000 7f306517c640 1 -- 172.21.15.5:0/689843518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30600c9a70 con 0x7f304c005bb0 2023-11-24T06:17:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.651+0000 7f306517c640 1 -- 172.21.15.5:0/689843518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30600c8de0 con 0x7f304c005bb0 2023-11-24T06:17:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.652+0000 7f306517c640 1 -- 172.21.15.5:0/689843518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30600c9c00 con 0x7f304c005bb0 2023-11-24T06:17:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.653+0000 7f30577fe640 1 -- 172.21.15.5:0/689843518 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3050031030 con 0x7f304c005bb0 2023-11-24T06:17:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.653+0000 7f30577fe640 1 --2- 172.21.15.5:0/689843518 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f304006aec0 0x7f304006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.653+0000 7f30577fe640 1 -- 172.21.15.5:0/689843518 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f30500ab2a0 con 0x7f304c005bb0 2023-11-24T06:17:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.654+0000 7f305e59c640 1 --2- 172.21.15.5:0/689843518 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f304006aec0 0x7f304006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:11.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.654+0000 7f305e59c640 1 --2- 172.21.15.5:0/689843518 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f304006aec0 0x7f304006d380 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3048006f00 tx=0x7f3048006e10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:11.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.658+0000 7f30577fe640 1 -- 172.21.15.5:0/689843518 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3050076810 con 0x7f304c005bb0 2023-11-24T06:17:11.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.911+0000 7f306517c640 1 -- 172.21.15.5:0/689843518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f3060074770 con 0x7f304c005bb0 2023-11-24T06:17:11.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.915+0000 7f30577fe640 1 -- 172.21.15.5:0/689843518 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3050079e30 con 0x7f304c005bb0 2023-11-24T06:17:11.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.917+0000 7f306517c640 1 -- 172.21.15.5:0/689843518 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f304006aec0 msgr2=0x7f304006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:11.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.917+0000 7f306517c640 1 --2- 172.21.15.5:0/689843518 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f304006aec0 0x7f304006d380 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3048006f00 tx=0x7f3048006e10 comp rx=0 tx=0).stop 2023-11-24T06:17:11.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.917+0000 7f306517c640 1 -- 172.21.15.5:0/689843518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f304c005bb0 msgr2=0x7f30601f2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:11.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.917+0000 7f306517c640 1 --2- 172.21.15.5:0/689843518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f304c005bb0 0x7f30601f2fb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3050000f80 tx=0x7f3050001cb0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.917+0000 7f306517c640 1 -- 172.21.15.5:0/689843518 shutdown_connections 2023-11-24T06:17:11.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.918+0000 7f306517c640 1 --2- 172.21.15.5:0/689843518 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f304006aec0 0x7f304006d380 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.918+0000 7f306517c640 1 --2- 172.21.15.5:0/689843518 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f30600cab70 0x7f30600cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.918+0000 7f306517c640 1 --2- 172.21.15.5:0/689843518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f30600ca6f0 0x7f30600ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.918+0000 7f306517c640 1 --2- 172.21.15.5:0/689843518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f304c005bb0 0x7f30601f2fb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:11.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.918+0000 7f306517c640 1 -- 172.21.15.5:0/689843518 >> 172.21.15.5:0/689843518 conn(0x7f30600b0d80 msgr2=0x7f306006d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:11.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.918+0000 7f306517c640 1 -- 172.21.15.5:0/689843518 shutdown_connections 2023-11-24T06:17:11.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:11.918+0000 7f306517c640 1 -- 172.21.15.5:0/689843518 wait complete. 2023-11-24T06:17:11.937 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-24T06:17:12.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.157+0000 7f64522e0640 1 Processor -- start 2023-11-24T06:17:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.158+0000 7f64522e0640 1 -- start start 2023-11-24T06:17:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.159+0000 7f64522e0640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f644c06dc30 0x7f644c071090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.159+0000 7f64522e0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f644c0715d0 0x7f644c0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.159+0000 7f64522e0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644c0ad400 con 0x7f644c0b3b60 2023-11-24T06:17:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.159+0000 7f64522e0640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f644c0acb50 con 0x7f644c0715d0 2023-11-24T06:17:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.159+0000 7f64522e0640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f644c0acdf0 con 0x7f644c06dc30 2023-11-24T06:17:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.160+0000 7f645087d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f644c0715d0 0x7f644c0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.160+0000 7f644bfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f644c0b3b60 0x7f644c06d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59168/0 (socket says 172.21.15.5:59168) 2023-11-24T06:17:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.160+0000 7f644bfff640 1 -- 172.21.15.5:0/2226697444 learned_addr learned my addr 172.21.15.5:0/2226697444 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:12.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.160+0000 7f6443fff640 1 --2- 172.21.15.5:0/2226697444 >> v2:172.21.15.6:3300/0 conn(0x7f644c06dc30 0x7f644c071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:12.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.160+0000 7f645087d640 1 -- 172.21.15.5:0/2226697444 >> v2:172.21.15.6:3300/0 conn(0x7f644c06dc30 msgr2=0x7f644c071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:12.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.160+0000 7f645087d640 1 --2- 172.21.15.5:0/2226697444 >> v2:172.21.15.6:3300/0 conn(0x7f644c06dc30 0x7f644c071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.160+0000 7f645087d640 1 -- 172.21.15.5:0/2226697444 >> v1:172.21.15.5:6789/0 conn(0x7f644c0b3b60 legacy=0x7f644c06d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:12.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.160+0000 7f645087d640 1 -- 172.21.15.5:0/2226697444 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c0720a0 con 0x7f644c0715d0 2023-11-24T06:17:12.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.160+0000 7f6443fff640 1 --2- 172.21.15.5:0/2226697444 >> v2:172.21.15.6:3300/0 conn(0x7f644c06dc30 0x7f644c071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.161+0000 7f645087d640 1 --2- 172.21.15.5:0/2226697444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644c0715d0 0x7f644c0ad890 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f644c06bdb0 tx=0x7f644402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa07d9a84ae7827 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.162+0000 7f644b7fe640 1 -- 172.21.15.5:0/2226697444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644402cc70 con 0x7f644c0715d0 2023-11-24T06:17:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.162+0000 7f644b7fe640 1 -- 172.21.15.5:0/2226697444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644402cdd0 con 0x7f644c0715d0 2023-11-24T06:17:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.162+0000 7f644b7fe640 1 -- 172.21.15.5:0/2226697444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444034660 con 0x7f644c0715d0 2023-11-24T06:17:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.162+0000 7f64522e0640 1 -- 172.21.15.5:0/2226697444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644c0715d0 msgr2=0x7f644c0ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.162+0000 7f64522e0640 1 --2- 172.21.15.5:0/2226697444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644c0715d0 0x7f644c0ad890 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f644c06bdb0 tx=0x7f644402bec0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.162+0000 7f64522e0640 1 -- 172.21.15.5:0/2226697444 shutdown_connections 2023-11-24T06:17:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.162+0000 7f64522e0640 1 --2- 172.21.15.5:0/2226697444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644c0715d0 0x7f644c0ad890 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.162+0000 7f64522e0640 1 --2- 172.21.15.5:0/2226697444 >> v2:172.21.15.6:3300/0 conn(0x7f644c06dc30 0x7f644c071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.162+0000 7f64522e0640 1 -- 172.21.15.5:0/2226697444 >> 172.21.15.5:0/2226697444 conn(0x7f644c0b0d80 msgr2=0x7f644c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.162+0000 7f64522e0640 1 -- 172.21.15.5:0/2226697444 shutdown_connections 2023-11-24T06:17:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.162+0000 7f64522e0640 1 -- 172.21.15.5:0/2226697444 wait complete. 2023-11-24T06:17:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.164+0000 7f64522e0640 1 Processor -- start 2023-11-24T06:17:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.164+0000 7f64522e0640 1 -- start start 2023-11-24T06:17:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.165+0000 7f64522e0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f644c06dc30 0x7f644c161650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.165+0000 7f64522e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644c0715d0 0x7f644c159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.165+0000 7f64522e0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f644c0b3b60 0x7f644c14c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.165+0000 7f64522e0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c1596c0 con 0x7f644c0715d0 2023-11-24T06:17:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.165+0000 7f64522e0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c1598f0 con 0x7f644c06dc30 2023-11-24T06:17:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.165+0000 7f64522e0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c159bd0 con 0x7f644c0b3b60 2023-11-24T06:17:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.165+0000 7f6443fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644c0715d0 0x7f644c159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.166+0000 7f6443fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644c0715d0 0x7f644c159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53648/0 (socket says 172.21.15.5:53648) 2023-11-24T06:17:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.166+0000 7f6443fff640 1 -- 172.21.15.5:0/716995520 learned_addr learned my addr 172.21.15.5:0/716995520 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.166+0000 7f644bfff640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f644c06dc30 0x7f644c161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.166+0000 7f645087d640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f644c0b3b60 0x7f644c14c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.166+0000 7f6443fff640 1 -- 172.21.15.5:0/716995520 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f644c0b3b60 msgr2=0x7f644c14c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.166+0000 7f6443fff640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f644c0b3b60 0x7f644c14c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.166+0000 7f6443fff640 1 -- 172.21.15.5:0/716995520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f644c06dc30 msgr2=0x7f644c161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.166+0000 7f6443fff640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f644c06dc30 0x7f644c161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.166+0000 7f6443fff640 1 -- 172.21.15.5:0/716995520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6444009a70 con 0x7f644c0715d0 2023-11-24T06:17:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.166+0000 7f645087d640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f644c0b3b60 0x7f644c14c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.166+0000 7f6443fff640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644c0715d0 0x7f644c159180 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f643400c7f0 tx=0x7f643400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.167+0000 7f64497fa640 1 -- 172.21.15.5:0/716995520 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434007820 con 0x7f644c0715d0 2023-11-24T06:17:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.167+0000 7f64497fa640 1 -- 172.21.15.5:0/716995520 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6434007980 con 0x7f644c0715d0 2023-11-24T06:17:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.167+0000 7f64522e0640 1 -- 172.21.15.5:0/716995520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644c159eb0 con 0x7f644c0715d0 2023-11-24T06:17:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.167+0000 7f64497fa640 1 -- 172.21.15.5:0/716995520 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434007cf0 con 0x7f644c0715d0 2023-11-24T06:17:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.167+0000 7f64522e0640 1 -- 172.21.15.5:0/716995520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644c15a470 con 0x7f644c0715d0 2023-11-24T06:17:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.168+0000 7f64522e0640 1 -- 172.21.15.5:0/716995520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644c074770 con 0x7f644c0715d0 2023-11-24T06:17:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.169+0000 7f64497fa640 1 -- 172.21.15.5:0/716995520 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f64340113f0 con 0x7f644c0715d0 2023-11-24T06:17:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.170+0000 7f64497fa640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f642c06ae40 0x7f642c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.170+0000 7f64497fa640 1 -- 172.21.15.5:0/716995520 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6434089670 con 0x7f644c0715d0 2023-11-24T06:17:12.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.170+0000 7f644bfff640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f642c06ae40 0x7f642c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:12.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.171+0000 7f644bfff640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f642c06ae40 0x7f642c06d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f643c005880 tx=0x7f643c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:12.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.175+0000 7f64497fa640 1 -- 172.21.15.5:0/716995520 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f64340549d0 con 0x7f644c0715d0 2023-11-24T06:17:12.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.443+0000 7f64522e0640 1 -- 172.21.15.5:0/716995520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f644c06e320 con 0x7f644c0715d0 2023-11-24T06:17:12.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.448+0000 7f64497fa640 1 -- 172.21.15.5:0/716995520 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6434057ff0 con 0x7f644c0715d0 2023-11-24T06:17:12.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.451+0000 7f64522e0640 1 -- 172.21.15.5:0/716995520 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f642c06ae40 msgr2=0x7f642c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:12.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.451+0000 7f64522e0640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f642c06ae40 0x7f642c06d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f643c005880 tx=0x7f643c009040 comp rx=0 tx=0).stop 2023-11-24T06:17:12.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.451+0000 7f64522e0640 1 -- 172.21.15.5:0/716995520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644c0715d0 msgr2=0x7f644c159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:12.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.451+0000 7f64522e0640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644c0715d0 0x7f644c159180 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f643400c7f0 tx=0x7f643400ccb0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.451+0000 7f64522e0640 1 -- 172.21.15.5:0/716995520 shutdown_connections 2023-11-24T06:17:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.451+0000 7f64522e0640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f642c06ae40 0x7f642c06d300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.451+0000 7f64522e0640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f644c0b3b60 0x7f644c14c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.451+0000 7f64522e0640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644c0715d0 0x7f644c159180 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.451+0000 7f64522e0640 1 --2- 172.21.15.5:0/716995520 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f644c06dc30 0x7f644c161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.451+0000 7f64522e0640 1 -- 172.21.15.5:0/716995520 >> 172.21.15.5:0/716995520 conn(0x7f644c0b0d80 msgr2=0x7f644c0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.451+0000 7f64522e0640 1 -- 172.21.15.5:0/716995520 shutdown_connections 2023-11-24T06:17:12.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.452+0000 7f64522e0640 1 -- 172.21.15.5:0/716995520 wait complete. 2023-11-24T06:17:12.473 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-24T06:17:12.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.698+0000 7fa8620ef640 1 Processor -- start 2023-11-24T06:17:12.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.698+0000 7fa8620ef640 1 -- start start 2023-11-24T06:17:12.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.700+0000 7fa8620ef640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa85c0d3810 0x7fa85c0cfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:12.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.700+0000 7fa8620ef640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85c15d810 con 0x7fa85c0d1690 2023-11-24T06:17:12.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.700+0000 7fa8620ef640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85c0d0130 con 0x7fa85c15d430 2023-11-24T06:17:12.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.700+0000 7fa8620ef640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa85c0d09d0 con 0x7fa85c0d3810 2023-11-24T06:17:12.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.701+0000 7fa8618ee640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa85c0d1690 0x7fa85c0cdbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:59180/0 (socket says 172.21.15.5:59180) 2023-11-24T06:17:12.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.701+0000 7fa8618ee640 1 -- 172.21.15.5:0/3555857038 learned_addr learned my addr 172.21.15.5:0/3555857038 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:12.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.701+0000 7fa8608ec640 1 --2- 172.21.15.5:0/3555857038 >> v2:172.21.15.6:3300/0 conn(0x7fa85c0d3810 0x7fa85c0cfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:12.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.701+0000 7fa853fff640 1 -- 172.21.15.5:0/3555857038 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223696509 0 0) 0x7fa85c15d810 con 0x7fa85c0d1690 2023-11-24T06:17:12.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.701+0000 7fa853fff640 1 -- 172.21.15.5:0/3555857038 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa848003520 con 0x7fa85c0d1690 2023-11-24T06:17:12.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.701+0000 7fa853fff640 1 -- 172.21.15.5:0/3555857038 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 143746351 0 0) 0x7fa848003520 con 0x7fa85c0d1690 2023-11-24T06:17:12.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.701+0000 7fa853fff640 1 -- 172.21.15.5:0/3555857038 >> v2:172.21.15.6:3300/0 conn(0x7fa85c0d3810 msgr2=0x7fa85c0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:12.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.701+0000 7fa853fff640 1 --2- 172.21.15.5:0/3555857038 >> v2:172.21.15.6:3300/0 conn(0x7fa85c0d3810 0x7fa85c0cfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.701+0000 7fa853fff640 1 -- 172.21.15.5:0/3555857038 >> v1:172.21.15.6:6789/0 conn(0x7fa85c15d430 legacy=0x7fa85c1615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:12.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.701+0000 7fa853fff640 1 -- 172.21.15.5:0/3555857038 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c0d0410 con 0x7fa85c0d1690 2023-11-24T06:17:12.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.701+0000 7fa853fff640 1 -- 172.21.15.5:0/3555857038 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fa854001670 con 0x7fa85c0d1690 2023-11-24T06:17:12.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa853fff640 1 -- 172.21.15.5:0/3555857038 >> v1:172.21.15.5:6789/0 conn(0x7fa85c0d1690 legacy=0x7fa85c0cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:12.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa853fff640 1 --2- 172.21.15.5:0/3555857038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8480039e0 0x7fa848003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:12.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa853fff640 1 --2- 172.21.15.5:0/3555857038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8480047c0 0x7fa848004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:12.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa853fff640 1 --2- 172.21.15.5:0/3555857038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa848005180 0x7fa84800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:12.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa853fff640 1 -- 172.21.15.5:0/3555857038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c0d09d0 con 0x7fa8480039e0 2023-11-24T06:17:12.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa853fff640 1 -- 172.21.15.5:0/3555857038 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84800c0b0 con 0x7fa848005180 2023-11-24T06:17:12.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa853fff640 1 -- 172.21.15.5:0/3555857038 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84800c270 con 0x7fa8480047c0 2023-11-24T06:17:12.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa8610ed640 1 --2- 172.21.15.5:0/3555857038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8480039e0 0x7fa848003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:12.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa8608ec640 1 --2- 172.21.15.5:0/3555857038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8480047c0 0x7fa848004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:12.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa8610ed640 1 -- 172.21.15.5:0/3555857038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8480047c0 msgr2=0x7fa848004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:12.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa8610ed640 1 --2- 172.21.15.5:0/3555857038 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa8480047c0 0x7fa848004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa8610ed640 1 -- 172.21.15.5:0/3555857038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa848005180 msgr2=0x7fa84800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:12.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa8610ed640 1 --2- 172.21.15.5:0/3555857038 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa848005180 0x7fa84800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa8610ed640 1 -- 172.21.15.5:0/3555857038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa848003520 con 0x7fa8480039e0 2023-11-24T06:17:12.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.702+0000 7fa8618ee640 1 -- 172.21.15.5:0/3555857038 reap_dead start 2023-11-24T06:17:12.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.703+0000 7fa8610ed640 1 --2- 172.21.15.5:0/3555857038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8480039e0 0x7fa848003e00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa858001e50 tx=0x7fa8480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7767c7329f1dd44 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:12.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.704+0000 7fa853fff640 1 -- 172.21.15.5:0/3555857038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858003b70 con 0x7fa8480039e0 2023-11-24T06:17:12.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.704+0000 7fa853fff640 1 -- 172.21.15.5:0/3555857038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa858003cd0 con 0x7fa8480039e0 2023-11-24T06:17:12.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.704+0000 7fa853fff640 1 -- 172.21.15.5:0/3555857038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858005310 con 0x7fa8480039e0 2023-11-24T06:17:12.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.704+0000 7fa8620ef640 1 -- 172.21.15.5:0/3555857038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8480039e0 msgr2=0x7fa848003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:12.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.704+0000 7fa8620ef640 1 --2- 172.21.15.5:0/3555857038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8480039e0 0x7fa848003e00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa858001e50 tx=0x7fa8480057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.705+0000 7fa8620ef640 1 -- 172.21.15.5:0/3555857038 shutdown_connections 2023-11-24T06:17:12.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.705+0000 7fa8620ef640 1 --2- 172.21.15.5:0/3555857038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8480039e0 0x7fa848003e00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.705+0000 7fa8620ef640 1 -- 172.21.15.5:0/3555857038 >> 172.21.15.5:0/3555857038 conn(0x7fa85c08cb00 msgr2=0x7fa85c09c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:12.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.705+0000 7fa8620ef640 1 -- 172.21.15.5:0/3555857038 shutdown_connections 2023-11-24T06:17:12.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.705+0000 7fa8620ef640 1 -- 172.21.15.5:0/3555857038 wait complete. 2023-11-24T06:17:12.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.707+0000 7fa8620ef640 1 Processor -- start 2023-11-24T06:17:12.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.707+0000 7fa8620ef640 1 -- start start 2023-11-24T06:17:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8620ef640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8480039e0 0x7fa85c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8620ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa85c151c50 0x7fa85c1520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8620ef640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa85c157e90 0x7fa85c158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8620ef640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c0d3ca0 con 0x7fa85c151c50 2023-11-24T06:17:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8620ef640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c153b70 con 0x7fa8480039e0 2023-11-24T06:17:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8620ef640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c152ef0 con 0x7fa85c157e90 2023-11-24T06:17:12.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8608ec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa85c151c50 0x7fa85c1520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8608ec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa85c151c50 0x7fa85c1520d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53662/0 (socket says 172.21.15.5:53662) 2023-11-24T06:17:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8608ec640 1 -- 172.21.15.5:0/3983679292 learned_addr learned my addr 172.21.15.5:0/3983679292 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8610ed640 1 --2- 172.21.15.5:0/3983679292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8480039e0 0x7fa85c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8608ec640 1 -- 172.21.15.5:0/3983679292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa85c157e90 msgr2=0x7fa85c158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8608ec640 1 --2- 172.21.15.5:0/3983679292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa85c157e90 0x7fa85c158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8608ec640 1 -- 172.21.15.5:0/3983679292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8480039e0 msgr2=0x7fa85c0b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8608ec640 1 --2- 172.21.15.5:0/3983679292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8480039e0 0x7fa85c0b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8608ec640 1 -- 172.21.15.5:0/3983679292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c1531d0 con 0x7fa85c151c50 2023-11-24T06:17:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.708+0000 7fa8610ed640 1 --2- 172.21.15.5:0/3983679292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8480039e0 0x7fa85c0b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.709+0000 7fa8608ec640 1 --2- 172.21.15.5:0/3983679292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa85c151c50 0x7fa85c1520d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa84c010790 tx=0x7fa84c010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.709+0000 7fa851ffb640 1 -- 172.21.15.5:0/3983679292 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c009630 con 0x7fa85c151c50 2023-11-24T06:17:12.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.709+0000 7fa851ffb640 1 -- 172.21.15.5:0/3983679292 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84c00eb60 con 0x7fa85c151c50 2023-11-24T06:17:12.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.709+0000 7fa851ffb640 1 -- 172.21.15.5:0/3983679292 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c010ea0 con 0x7fa85c151c50 2023-11-24T06:17:12.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.709+0000 7fa8620ef640 1 -- 172.21.15.5:0/3983679292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa85c1534b0 con 0x7fa85c151c50 2023-11-24T06:17:12.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.709+0000 7fa8620ef640 1 -- 172.21.15.5:0/3983679292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa85c152610 con 0x7fa85c151c50 2023-11-24T06:17:12.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.710+0000 7fa8620ef640 1 -- 172.21.15.5:0/3983679292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85c067960 con 0x7fa85c151c50 2023-11-24T06:17:12.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.711+0000 7fa851ffb640 1 -- 172.21.15.5:0/3983679292 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa84c01c400 con 0x7fa85c151c50 2023-11-24T06:17:12.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.713+0000 7fa851ffb640 1 --2- 172.21.15.5:0/3983679292 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa84006ae40 0x7fa84006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:12.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.713+0000 7fa851ffb640 1 -- 172.21.15.5:0/3983679292 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa84c089770 con 0x7fa85c151c50 2023-11-24T06:17:12.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.713+0000 7fa8610ed640 1 --2- 172.21.15.5:0/3983679292 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa84006ae40 0x7fa84006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:12.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.714+0000 7fa8610ed640 1 --2- 172.21.15.5:0/3983679292 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa84006ae40 0x7fa84006d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa858034000 tx=0x7fa858009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:12.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.718+0000 7fa851ffb640 1 -- 172.21.15.5:0/3983679292 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa84c054b60 con 0x7fa85c151c50 2023-11-24T06:17:12.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.979+0000 7fa8620ef640 1 -- 172.21.15.5:0/3983679292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fa85c04e840 con 0x7fa85c151c50 2023-11-24T06:17:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.984+0000 7fa851ffb640 1 -- 172.21.15.5:0/3983679292 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa84c058180 con 0x7fa85c151c50 2023-11-24T06:17:12.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.987+0000 7fa8620ef640 1 -- 172.21.15.5:0/3983679292 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa84006ae40 msgr2=0x7fa84006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:12.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.987+0000 7fa8620ef640 1 --2- 172.21.15.5:0/3983679292 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa84006ae40 0x7fa84006d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa858034000 tx=0x7fa858009030 comp rx=0 tx=0).stop 2023-11-24T06:17:12.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.987+0000 7fa8620ef640 1 -- 172.21.15.5:0/3983679292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa85c151c50 msgr2=0x7fa85c1520d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:12.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.987+0000 7fa8620ef640 1 --2- 172.21.15.5:0/3983679292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa85c151c50 0x7fa85c1520d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa84c010790 tx=0x7fa84c010c50 comp rx=0 tx=0).stop 2023-11-24T06:17:12.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.987+0000 7fa8620ef640 1 -- 172.21.15.5:0/3983679292 shutdown_connections 2023-11-24T06:17:12.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.987+0000 7fa8620ef640 1 --2- 172.21.15.5:0/3983679292 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa84006ae40 0x7fa84006d300 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.987+0000 7fa8620ef640 1 --2- 172.21.15.5:0/3983679292 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa85c157e90 0x7fa85c158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.987+0000 7fa8620ef640 1 --2- 172.21.15.5:0/3983679292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa85c151c50 0x7fa85c1520d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.987+0000 7fa8620ef640 1 --2- 172.21.15.5:0/3983679292 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa8480039e0 0x7fa85c0b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:12.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.987+0000 7fa8620ef640 1 -- 172.21.15.5:0/3983679292 >> 172.21.15.5:0/3983679292 conn(0x7fa85c08cb00 msgr2=0x7fa85c0cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:12.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.987+0000 7fa8620ef640 1 -- 172.21.15.5:0/3983679292 shutdown_connections 2023-11-24T06:17:12.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:12.987+0000 7fa8620ef640 1 -- 172.21.15.5:0/3983679292 wait complete. 2023-11-24T06:17:13.007 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:17:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.217+0000 7f3a9d5fa640 1 Processor -- start 2023-11-24T06:17:13.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.217+0000 7f3a9d5fa640 1 -- start start 2023-11-24T06:17:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.219+0000 7f3a9d5fa640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f3a9813dfd0 0x7f3a9815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.219+0000 7f3a9d5fa640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a980cf050 0x7f3a980cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.219+0000 7f3a9d5fa640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a980ce7c0 con 0x7f3a9806c130 2023-11-24T06:17:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.219+0000 7f3a9d5fa640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a980cfa40 con 0x7f3a980cf050 2023-11-24T06:17:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.219+0000 7f3a9d5fa640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a98140900 con 0x7f3a9813dfd0 2023-11-24T06:17:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.219+0000 7f3a8f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a980cf050 0x7f3a980cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.219+0000 7f3a8f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3a980cf050 0x7f3a980cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50274/0 (socket says 172.21.15.5:50274) 2023-11-24T06:17:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.219+0000 7f3a8f7fe640 1 -- 172.21.15.5:0/3952279102 learned_addr learned my addr 172.21.15.5:0/3952279102 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.220+0000 7f3a8f7fe640 1 -- 172.21.15.5:0/3952279102 >> v2:172.21.15.6:3300/0 conn(0x7f3a9813dfd0 msgr2=0x7f3a9815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.220+0000 7f3a8f7fe640 1 --2- 172.21.15.5:0/3952279102 >> v2:172.21.15.6:3300/0 conn(0x7f3a9813dfd0 0x7f3a9815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.220+0000 7f3a8f7fe640 1 -- 172.21.15.5:0/3952279102 >> v1:172.21.15.6:6789/0 conn(0x7f3a9806c130 legacy=0x7f3a9806c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.220+0000 7f3a8f7fe640 1 -- 172.21.15.5:0/3952279102 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9815fa70 con 0x7f3a980cf050 2023-11-24T06:17:13.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.221+0000 7f3a8f7fe640 1 --2- 172.21.15.5:0/3952279102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a980cf050 0x7f3a980cf500 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3a980cfc10 tx=0x7f3a8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a44d370f019a6ea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:13.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.221+0000 7f3a977fe640 1 -- 172.21.15.5:0/3952279102 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80031870 con 0x7f3a980cf050 2023-11-24T06:17:13.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.221+0000 7f3a977fe640 1 -- 172.21.15.5:0/3952279102 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a800319d0 con 0x7f3a980cf050 2023-11-24T06:17:13.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.221+0000 7f3a977fe640 1 -- 172.21.15.5:0/3952279102 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80031d60 con 0x7f3a980cf050 2023-11-24T06:17:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.222+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/3952279102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a980cf050 msgr2=0x7f3a980cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.222+0000 7f3a9d5fa640 1 --2- 172.21.15.5:0/3952279102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a980cf050 0x7f3a980cf500 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3a980cfc10 tx=0x7f3a8002b910 comp rx=0 tx=0).stop 2023-11-24T06:17:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.222+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/3952279102 shutdown_connections 2023-11-24T06:17:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.222+0000 7f3a9d5fa640 1 --2- 172.21.15.5:0/3952279102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a980cf050 0x7f3a980cf500 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.222+0000 7f3a9d5fa640 1 --2- 172.21.15.5:0/3952279102 >> v2:172.21.15.6:3300/0 conn(0x7f3a9813dfd0 0x7f3a9815be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.222+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/3952279102 >> 172.21.15.5:0/3952279102 conn(0x7f3a9806a860 msgr2=0x7f3a9806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.222+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/3952279102 shutdown_connections 2023-11-24T06:17:13.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.222+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/3952279102 wait complete. 2023-11-24T06:17:13.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.224+0000 7f3a9d5fa640 1 Processor -- start 2023-11-24T06:17:13.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.224+0000 7f3a9d5fa640 1 -- start start 2023-11-24T06:17:13.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.225+0000 7f3a9d5fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9806c130 0x7f3a981553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:13.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.225+0000 7f3a9d5fa640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a980cf050 0x7f3a98147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:13.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.225+0000 7f3a9d5fa640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a9813dfd0 0x7f3a98147d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:13.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.225+0000 7f3a9d5fa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a981558f0 con 0x7f3a9806c130 2023-11-24T06:17:13.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.225+0000 7f3a9d5fa640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a98159cc0 con 0x7f3a9813dfd0 2023-11-24T06:17:13.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.225+0000 7f3a9d5fa640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a98159fa0 con 0x7f3a980cf050 2023-11-24T06:17:13.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.225+0000 7f3a97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9806c130 0x7f3a981553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:13.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.225+0000 7f3a97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9806c130 0x7f3a981553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50288/0 (socket says 172.21.15.5:50288) 2023-11-24T06:17:13.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.225+0000 7f3a97fff640 1 -- 172.21.15.5:0/438393179 learned_addr learned my addr 172.21.15.5:0/438393179 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:13.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.225+0000 7f3a9cdf9640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a9813dfd0 0x7f3a98147d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:13.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.226+0000 7f3a8f7fe640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a980cf050 0x7f3a98147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.225+0000 7f3a97fff640 1 -- 172.21.15.5:0/438393179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a980cf050 msgr2=0x7f3a98147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.226+0000 7f3a97fff640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a980cf050 0x7f3a98147810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.226+0000 7f3a97fff640 1 -- 172.21.15.5:0/438393179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a9813dfd0 msgr2=0x7f3a98147d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.226+0000 7f3a97fff640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a9813dfd0 0x7f3a98147d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.226+0000 7f3a97fff640 1 -- 172.21.15.5:0/438393179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a800094d0 con 0x7f3a9806c130 2023-11-24T06:17:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.226+0000 7f3a9cdf9640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a9813dfd0 0x7f3a98147d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.226+0000 7f3a97fff640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9806c130 0x7f3a981553b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800ba50 tx=0x7f3a8800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.226+0000 7f3a957fa640 1 -- 172.21.15.5:0/438393179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8800cbe0 con 0x7f3a9806c130 2023-11-24T06:17:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.226+0000 7f3a957fa640 1 -- 172.21.15.5:0/438393179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8800cd60 con 0x7f3a9806c130 2023-11-24T06:17:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.226+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/438393179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9815a280 con 0x7f3a9806c130 2023-11-24T06:17:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.226+0000 7f3a957fa640 1 -- 172.21.15.5:0/438393179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88014680 con 0x7f3a9806c130 2023-11-24T06:17:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.226+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/438393179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a98155dc0 con 0x7f3a9806c130 2023-11-24T06:17:13.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.228+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/438393179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9804e840 con 0x7f3a9806c130 2023-11-24T06:17:13.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.228+0000 7f3a957fa640 1 -- 172.21.15.5:0/438393179 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a8801d3b0 con 0x7f3a9806c130 2023-11-24T06:17:13.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.229+0000 7f3a957fa640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3a7806ae40 0x7f3a7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:13.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.230+0000 7f3a957fa640 1 -- 172.21.15.5:0/438393179 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3a88089220 con 0x7f3a9806c130 2023-11-24T06:17:13.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.230+0000 7f3a8f7fe640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3a7806ae40 0x7f3a7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:13.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.231+0000 7f3a8f7fe640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3a7806ae40 0x7f3a7806d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3a80036000 tx=0x7f3a8003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:13.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.234+0000 7f3a957fa640 1 -- 172.21.15.5:0/438393179 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a880546b0 con 0x7f3a9806c130 2023-11-24T06:17:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.430+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/438393179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a981e4d80 con 0x7f3a9806c130 2023-11-24T06:17:13.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.432+0000 7f3a957fa640 1 -- 172.21.15.5:0/438393179 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+12713 (secure 0 0 0) 0x7f3a88057cd0 con 0x7f3a9806c130 2023-11-24T06:17:13.437 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:13.437 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":31,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:17:09.361583+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:17:13.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.435+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/438393179 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3a7806ae40 msgr2=0x7f3a7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:13.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.435+0000 7f3a9d5fa640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3a7806ae40 0x7f3a7806d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3a80036000 tx=0x7f3a8003c040 comp rx=0 tx=0).stop 2023-11-24T06:17:13.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.435+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/438393179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9806c130 msgr2=0x7f3a981553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:13.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.435+0000 7f3a9d5fa640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9806c130 0x7f3a981553b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800ba50 tx=0x7f3a8800bf10 comp rx=0 tx=0).stop 2023-11-24T06:17:13.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.435+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/438393179 shutdown_connections 2023-11-24T06:17:13.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.435+0000 7f3a9d5fa640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f3a7806ae40 0x7f3a7806d300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.435+0000 7f3a9d5fa640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f3a980cf050 0x7f3a98147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.435+0000 7f3a9d5fa640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f3a9813dfd0 0x7f3a98147d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.435+0000 7f3a9d5fa640 1 --2- 172.21.15.5:0/438393179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a9806c130 0x7f3a981553b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.435+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/438393179 >> 172.21.15.5:0/438393179 conn(0x7f3a9806a860 msgr2=0x7f3a9815de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:13.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.435+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/438393179 shutdown_connections 2023-11-24T06:17:13.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.435+0000 7f3a9d5fa640 1 -- 172.21.15.5:0/438393179 wait complete. 2023-11-24T06:17:13.456 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:17:13.456 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:17:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.683+0000 7efef2490640 1 Processor -- start 2023-11-24T06:17:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.683+0000 7efef2490640 1 -- start start 2023-11-24T06:17:13.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.685+0000 7efef2490640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efeec15b350 0x7efeec0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:13.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.685+0000 7efef2490640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efeec09cb30 0x7efeec0b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:13.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.685+0000 7efef2490640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeec15b730 con 0x7efeec0adb60 2023-11-24T06:17:13.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.685+0000 7efef2490640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeec0ac740 con 0x7efeec15b350 2023-11-24T06:17:13.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.685+0000 7efef2490640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec0ac460 con 0x7efeec09cb30 2023-11-24T06:17:13.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.686+0000 7efeebfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efeec15b350 0x7efeec0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:13.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.686+0000 7efeebfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7efeec15b350 0x7efeec0abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50292/0 (socket says 172.21.15.5:50292) 2023-11-24T06:17:13.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.686+0000 7efeebfff640 1 -- 172.21.15.5:0/19244336 learned_addr learned my addr 172.21.15.5:0/19244336 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.686+0000 7efeebfff640 1 -- 172.21.15.5:0/19244336 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efeec09cb30 msgr2=0x7efeec0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.686+0000 7efeebfff640 1 --2- 172.21.15.5:0/19244336 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efeec09cb30 0x7efeec0b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.686+0000 7efeebfff640 1 -- 172.21.15.5:0/19244336 >> v1:172.21.15.6:6789/0 conn(0x7efeec0adb60 legacy=0x7efeec0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.686+0000 7efeebfff640 1 -- 172.21.15.5:0/19244336 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec0b2a50 con 0x7efeec15b350 2023-11-24T06:17:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.687+0000 7efeebfff640 1 --2- 172.21.15.5:0/19244336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efeec15b350 0x7efeec0abe80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efedc009870 tx=0x7efedc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8820d84b282f3ddb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:13.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.688+0000 7efeeb7fe640 1 -- 172.21.15.5:0/19244336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc031640 con 0x7efeec15b350 2023-11-24T06:17:13.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.688+0000 7efeeb7fe640 1 -- 172.21.15.5:0/19244336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc0317a0 con 0x7efeec15b350 2023-11-24T06:17:13.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.688+0000 7efeeb7fe640 1 -- 172.21.15.5:0/19244336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc031b30 con 0x7efeec15b350 2023-11-24T06:17:13.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.688+0000 7efef2490640 1 -- 172.21.15.5:0/19244336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efeec15b350 msgr2=0x7efeec0abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.688+0000 7efef2490640 1 --2- 172.21.15.5:0/19244336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efeec15b350 0x7efeec0abe80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efedc009870 tx=0x7efedc02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.688+0000 7efef2490640 1 -- 172.21.15.5:0/19244336 shutdown_connections 2023-11-24T06:17:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.688+0000 7efef2490640 1 --2- 172.21.15.5:0/19244336 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efeec09cb30 0x7efeec0b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.688+0000 7efef2490640 1 --2- 172.21.15.5:0/19244336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efeec15b350 0x7efeec0abe80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.688+0000 7efef2490640 1 -- 172.21.15.5:0/19244336 >> 172.21.15.5:0/19244336 conn(0x7efeec0b0ba0 msgr2=0x7efeec0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.688+0000 7efef2490640 1 -- 172.21.15.5:0/19244336 shutdown_connections 2023-11-24T06:17:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.688+0000 7efef2490640 1 -- 172.21.15.5:0/19244336 wait complete. 2023-11-24T06:17:13.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.690+0000 7efef2490640 1 Processor -- start 2023-11-24T06:17:13.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.690+0000 7efef2490640 1 -- start start 2023-11-24T06:17:13.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.691+0000 7efef2490640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efeec09cb30 0x7efeec1f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.691+0000 7efef2490640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efeec0adb60 0x7efeec147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.691+0000 7efef2490640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efeec15b350 0x7efeec06da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.691+0000 7efef2490640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec14dc00 con 0x7efeec15b350 2023-11-24T06:17:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.691+0000 7efef2490640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeec1f3170 con 0x7efeec09cb30 2023-11-24T06:17:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.691+0000 7efef2490640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeec148690 con 0x7efeec0adb60 2023-11-24T06:17:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.692+0000 7efef0a2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efeec15b350 0x7efeec06da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.692+0000 7efef0a2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efeec15b350 0x7efeec06da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50298/0 (socket says 172.21.15.5:50298) 2023-11-24T06:17:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.692+0000 7efef0a2d640 1 -- 172.21.15.5:0/2036038170 learned_addr learned my addr 172.21.15.5:0/2036038170 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.692+0000 7efeebfff640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efeec09cb30 0x7efeec1f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.692+0000 7efee3fff640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efeec0adb60 0x7efeec147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.692+0000 7efef0a2d640 1 -- 172.21.15.5:0/2036038170 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efeec0adb60 msgr2=0x7efeec147cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.692+0000 7efef0a2d640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efeec0adb60 0x7efeec147cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.692+0000 7efef0a2d640 1 -- 172.21.15.5:0/2036038170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efeec09cb30 msgr2=0x7efeec1f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.692+0000 7efef0a2d640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efeec09cb30 0x7efeec1f2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.692+0000 7efef0a2d640 1 -- 172.21.15.5:0/2036038170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc0094d0 con 0x7efeec15b350 2023-11-24T06:17:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.692+0000 7efeebfff640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efeec09cb30 0x7efeec1f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.693+0000 7efef0a2d640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efeec15b350 0x7efeec06da30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efee400c780 tx=0x7efee400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.693+0000 7efee97fa640 1 -- 172.21.15.5:0/2036038170 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4003d80 con 0x7efeec15b350 2023-11-24T06:17:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.693+0000 7efee97fa640 1 -- 172.21.15.5:0/2036038170 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee400ce90 con 0x7efeec15b350 2023-11-24T06:17:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.693+0000 7efee97fa640 1 -- 172.21.15.5:0/2036038170 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee40052b0 con 0x7efeec15b350 2023-11-24T06:17:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.693+0000 7efef2490640 1 -- 172.21.15.5:0/2036038170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeec14a900 con 0x7efeec15b350 2023-11-24T06:17:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.693+0000 7efef2490640 1 -- 172.21.15.5:0/2036038170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeec14aec0 con 0x7efeec15b350 2023-11-24T06:17:13.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.694+0000 7efef2490640 1 -- 172.21.15.5:0/2036038170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeec09d200 con 0x7efeec15b350 2023-11-24T06:17:13.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.695+0000 7efee97fa640 1 -- 172.21.15.5:0/2036038170 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efee4005410 con 0x7efeec15b350 2023-11-24T06:17:13.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.696+0000 7efee97fa640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efecc06ae40 0x7efecc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:13.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.696+0000 7efee97fa640 1 -- 172.21.15.5:0/2036038170 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efee40887c0 con 0x7efeec15b350 2023-11-24T06:17:13.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.697+0000 7efeebfff640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efecc06ae40 0x7efecc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:13.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.697+0000 7efeebfff640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efecc06ae40 0x7efecc06d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efedc036000 tx=0x7efedc03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:13.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.701+0000 7efee97fa640 1 -- 172.21.15.5:0/2036038170 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efee4053b20 con 0x7efeec15b350 2023-11-24T06:17:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.893+0000 7efef2490640 1 -- 172.21.15.5:0/2036038170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efeec04e920 con 0x7efeec15b350 2023-11-24T06:17:13.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.894+0000 7efee97fa640 1 -- 172.21.15.5:0/2036038170 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7efee4057140 con 0x7efeec15b350 2023-11-24T06:17:13.896 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:17:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.896+0000 7efef2490640 1 -- 172.21.15.5:0/2036038170 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efecc06ae40 msgr2=0x7efecc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.896+0000 7efef2490640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efecc06ae40 0x7efecc06d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efedc036000 tx=0x7efedc03c040 comp rx=0 tx=0).stop 2023-11-24T06:17:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.896+0000 7efef2490640 1 -- 172.21.15.5:0/2036038170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efeec15b350 msgr2=0x7efeec06da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.896+0000 7efef2490640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efeec15b350 0x7efeec06da30 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7efee400c780 tx=0x7efee400cc40 comp rx=0 tx=0).stop 2023-11-24T06:17:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.897+0000 7efef2490640 1 -- 172.21.15.5:0/2036038170 shutdown_connections 2023-11-24T06:17:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.897+0000 7efef2490640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efecc06ae40 0x7efecc06d300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.897+0000 7efef2490640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efeec0adb60 0x7efeec147cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.897+0000 7efef2490640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efeec15b350 0x7efeec06da30 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.897+0000 7efef2490640 1 --2- 172.21.15.5:0/2036038170 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efeec09cb30 0x7efeec1f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.897+0000 7efef2490640 1 -- 172.21.15.5:0/2036038170 >> 172.21.15.5:0/2036038170 conn(0x7efeec0b0ba0 msgr2=0x7efeec1518b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.897+0000 7efef2490640 1 -- 172.21.15.5:0/2036038170 shutdown_connections 2023-11-24T06:17:13.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:13.897+0000 7efef2490640 1 -- 172.21.15.5:0/2036038170 wait complete. 2023-11-24T06:17:13.917 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T06:17:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.129+0000 7ff60f59e640 1 Processor -- start 2023-11-24T06:17:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.129+0000 7ff60f59e640 1 -- start start 2023-11-24T06:17:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.131+0000 7ff60f59e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6100cf050 0x7ff6100cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.131+0000 7ff60f59e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff610140900 con 0x7ff61013dfd0 2023-11-24T06:17:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.131+0000 7ff60f59e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61015fa70 con 0x7ff6100cd7b0 2023-11-24T06:17:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.131+0000 7ff60f59e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61015c580 con 0x7ff6100cf050 2023-11-24T06:17:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.131+0000 7ff60e59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff61013dfd0 0x7ff61015be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50818/0 (socket says 172.21.15.5:50818) 2023-11-24T06:17:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.131+0000 7ff60e59c640 1 -- 172.21.15.5:0/1100940187 learned_addr learned my addr 172.21.15.5:0/1100940187 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 -- 172.21.15.5:0/1100940187 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879641147 0 0) 0x7ff610140900 con 0x7ff61013dfd0 2023-11-24T06:17:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 -- 172.21.15.5:0/1100940187 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5f8003520 con 0x7ff61013dfd0 2023-11-24T06:17:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60dd9b640 1 --2- 172.21.15.5:0/1100940187 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6100cf050 0x7ff6100cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 -- 172.21.15.5:0/1100940187 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2522515340 0 0) 0x7ff5f8003520 con 0x7ff61013dfd0 2023-11-24T06:17:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 -- 172.21.15.5:0/1100940187 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6100cf050 msgr2=0x7ff6100cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 --2- 172.21.15.5:0/1100940187 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6100cf050 0x7ff6100cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 -- 172.21.15.5:0/1100940187 >> v1:172.21.15.6:6789/0 conn(0x7ff6100cd7b0 legacy=0x7ff6100cdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 -- 172.21.15.5:0/1100940187 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61015ee60 con 0x7ff61013dfd0 2023-11-24T06:17:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 -- 172.21.15.5:0/1100940187 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7ff600001670 con 0x7ff61013dfd0 2023-11-24T06:17:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 -- 172.21.15.5:0/1100940187 >> v1:172.21.15.5:6789/0 conn(0x7ff61013dfd0 legacy=0x7ff61015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 --2- 172.21.15.5:0/1100940187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5f8003a10 0x7ff5f8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 --2- 172.21.15.5:0/1100940187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f80047c0 0x7ff5f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 --2- 172.21.15.5:0/1100940187 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6100cf050 0x7ff5f8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 -- 172.21.15.5:0/1100940187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61015c580 con 0x7ff5f80047c0 2023-11-24T06:17:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 -- 172.21.15.5:0/1100940187 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f800ba80 con 0x7ff5f8003a10 2023-11-24T06:17:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.132+0000 7ff60d59a640 1 -- 172.21.15.5:0/1100940187 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f800bc40 con 0x7ff6100cf050 2023-11-24T06:17:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.133+0000 7ff60dd9b640 1 --2- 172.21.15.5:0/1100940187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f80047c0 0x7ff5f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.133+0000 7ff60e59c640 1 --2- 172.21.15.5:0/1100940187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5f8003a10 0x7ff5f8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.133+0000 7ff60ed9d640 1 --2- 172.21.15.5:0/1100940187 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6100cf050 0x7ff5f8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.133+0000 7ff60dd9b640 1 -- 172.21.15.5:0/1100940187 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6100cf050 msgr2=0x7ff5f8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.133+0000 7ff60dd9b640 1 --2- 172.21.15.5:0/1100940187 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff6100cf050 0x7ff5f8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.133+0000 7ff60dd9b640 1 -- 172.21.15.5:0/1100940187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5f8003a10 msgr2=0x7ff5f8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.133+0000 7ff60dd9b640 1 --2- 172.21.15.5:0/1100940187 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5f8003a10 0x7ff5f8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.134+0000 7ff60dd9b640 1 -- 172.21.15.5:0/1100940187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f8003520 con 0x7ff5f80047c0 2023-11-24T06:17:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.135+0000 7ff60dd9b640 1 --2- 172.21.15.5:0/1100940187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f80047c0 0x7ff5f8004c40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff604009ac0 tx=0x7ff5f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06246bca69ea049 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.135+0000 7ff60d59a640 1 -- 172.21.15.5:0/1100940187 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60402c6b0 con 0x7ff5f80047c0 2023-11-24T06:17:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.135+0000 7ff60d59a640 1 -- 172.21.15.5:0/1100940187 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60402c810 con 0x7ff5f80047c0 2023-11-24T06:17:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.135+0000 7ff60d59a640 1 -- 172.21.15.5:0/1100940187 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60402cba0 con 0x7ff5f80047c0 2023-11-24T06:17:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.136+0000 7ff60f59e640 1 -- 172.21.15.5:0/1100940187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f80047c0 msgr2=0x7ff5f8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.136+0000 7ff60f59e640 1 --2- 172.21.15.5:0/1100940187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f80047c0 0x7ff5f8004c40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff604009ac0 tx=0x7ff5f80057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.136+0000 7ff60ed9d640 1 -- 172.21.15.5:0/1100940187 reap_dead start 2023-11-24T06:17:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.136+0000 7ff60f59e640 1 -- 172.21.15.5:0/1100940187 shutdown_connections 2023-11-24T06:17:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.136+0000 7ff60f59e640 1 -- 172.21.15.5:0/1100940187 >> 172.21.15.5:0/1100940187 conn(0x7ff61006a860 msgr2=0x7ff61006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.136+0000 7ff60f59e640 1 -- 172.21.15.5:0/1100940187 shutdown_connections 2023-11-24T06:17:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.136+0000 7ff60f59e640 1 -- 172.21.15.5:0/1100940187 wait complete. 2023-11-24T06:17:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.138+0000 7ff60f59e640 1 Processor -- start 2023-11-24T06:17:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.138+0000 7ff60f59e640 1 -- start start 2023-11-24T06:17:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60f59e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5f80047c0 0x7ff610159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60f59e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff610133400 0x7ff610138b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6101390a0 0x7ff61013b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60f59e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff610139520 con 0x7ff6101390a0 2023-11-24T06:17:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60f59e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61015aa40 con 0x7ff610133400 2023-11-24T06:17:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60f59e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61015aed0 con 0x7ff5f80047c0 2023-11-24T06:17:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6101390a0 0x7ff61013b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6101390a0 0x7ff61013b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50310/0 (socket says 172.21.15.5:50310) 2023-11-24T06:17:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60ed9d640 1 -- 172.21.15.5:0/3063080229 learned_addr learned my addr 172.21.15.5:0/3063080229 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60e59c640 1 --2- 172.21.15.5:0/3063080229 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5f80047c0 0x7ff610159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60ed9d640 1 -- 172.21.15.5:0/3063080229 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5f80047c0 msgr2=0x7ff610159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60ed9d640 1 --2- 172.21.15.5:0/3063080229 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5f80047c0 0x7ff610159840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60ed9d640 1 -- 172.21.15.5:0/3063080229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff610133400 msgr2=0x7ff610138b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60ed9d640 1 --2- 172.21.15.5:0/3063080229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff610133400 0x7ff610138b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.139+0000 7ff60ed9d640 1 -- 172.21.15.5:0/3063080229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff610159e30 con 0x7ff6101390a0 2023-11-24T06:17:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.140+0000 7ff60ed9d640 1 --2- 172.21.15.5:0/3063080229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6101390a0 0x7ff61013b090 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff6100cfbe0 tx=0x7ff60800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.140+0000 7ff5feffd640 1 -- 172.21.15.5:0/3063080229 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608003970 con 0x7ff6101390a0 2023-11-24T06:17:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.140+0000 7ff5feffd640 1 -- 172.21.15.5:0/3063080229 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff608003ad0 con 0x7ff6101390a0 2023-11-24T06:17:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.140+0000 7ff5feffd640 1 -- 172.21.15.5:0/3063080229 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608003e80 con 0x7ff6101390a0 2023-11-24T06:17:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.140+0000 7ff60f59e640 1 -- 172.21.15.5:0/3063080229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61015a110 con 0x7ff6101390a0 2023-11-24T06:17:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.140+0000 7ff60f59e640 1 -- 172.21.15.5:0/3063080229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61015a6d0 con 0x7ff6101390a0 2023-11-24T06:17:14.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.141+0000 7ff60f59e640 1 -- 172.21.15.5:0/3063080229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61015a860 con 0x7ff6101390a0 2023-11-24T06:17:14.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.143+0000 7ff5feffd640 1 -- 172.21.15.5:0/3063080229 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff6080050d0 con 0x7ff6101390a0 2023-11-24T06:17:14.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.143+0000 7ff5feffd640 1 --2- 172.21.15.5:0/3063080229 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff5f006ae40 0x7ff5f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:14.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.144+0000 7ff5feffd640 1 -- 172.21.15.5:0/3063080229 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff608088fd0 con 0x7ff6101390a0 2023-11-24T06:17:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.144+0000 7ff60e59c640 1 --2- 172.21.15.5:0/3063080229 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff5f006ae40 0x7ff5f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.144+0000 7ff60e59c640 1 --2- 172.21.15.5:0/3063080229 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff5f006ae40 0x7ff5f006d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff600007c60 tx=0x7ff600009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:14.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.148+0000 7ff5feffd640 1 -- 172.21.15.5:0/3063080229 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff6080543d0 con 0x7ff6101390a0 2023-11-24T06:17:14.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.345+0000 7ff60f59e640 1 -- 172.21.15.5:0/3063080229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff61004e840 con 0x7ff6101390a0 2023-11-24T06:17:14.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.346+0000 7ff5feffd640 1 -- 172.21.15.5:0/3063080229 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff6080579f0 con 0x7ff6101390a0 2023-11-24T06:17:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:14.348 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2023-11-24T06:17:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2023-11-24T06:17:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2023-11-24T06:17:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:14.348 INFO:teuthology.orchestra.run.smithi005.stdout:] 2023-11-24T06:17:14.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.348+0000 7ff60f59e640 1 -- 172.21.15.5:0/3063080229 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff5f006ae40 msgr2=0x7ff5f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:14.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.348+0000 7ff60f59e640 1 --2- 172.21.15.5:0/3063080229 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff5f006ae40 0x7ff5f006d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff600007c60 tx=0x7ff600009040 comp rx=0 tx=0).stop 2023-11-24T06:17:14.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.348+0000 7ff60f59e640 1 -- 172.21.15.5:0/3063080229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6101390a0 msgr2=0x7ff61013b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:14.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.349+0000 7ff60f59e640 1 --2- 172.21.15.5:0/3063080229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6101390a0 0x7ff61013b090 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff6100cfbe0 tx=0x7ff60800ec10 comp rx=0 tx=0).stop 2023-11-24T06:17:14.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.349+0000 7ff60f59e640 1 -- 172.21.15.5:0/3063080229 shutdown_connections 2023-11-24T06:17:14.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.349+0000 7ff60f59e640 1 --2- 172.21.15.5:0/3063080229 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff5f006ae40 0x7ff5f006d300 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.349+0000 7ff60f59e640 1 --2- 172.21.15.5:0/3063080229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6101390a0 0x7ff61013b090 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.349+0000 7ff60f59e640 1 --2- 172.21.15.5:0/3063080229 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff610133400 0x7ff610138b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.349+0000 7ff60f59e640 1 --2- 172.21.15.5:0/3063080229 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5f80047c0 0x7ff610159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.349+0000 7ff60f59e640 1 -- 172.21.15.5:0/3063080229 >> 172.21.15.5:0/3063080229 conn(0x7ff61006a860 msgr2=0x7ff61013d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:14.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.349+0000 7ff60f59e640 1 -- 172.21.15.5:0/3063080229 shutdown_connections 2023-11-24T06:17:14.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.349+0000 7ff60f59e640 1 -- 172.21.15.5:0/3063080229 wait complete. 2023-11-24T06:17:14.370 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T06:17:14.371 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T06:17:14.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.601+0000 7f7a6ef41640 1 Processor -- start 2023-11-24T06:17:14.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.601+0000 7f7a6ef41640 1 -- start start 2023-11-24T06:17:14.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.603+0000 7f7a6ef41640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7a680acd70 0x7f7a6806dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:14.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.603+0000 7f7a6ef41640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a68070490 0x7f7a68070920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:14.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.603+0000 7f7a6ef41640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6806d7a0 con 0x7f7a680ae490 2023-11-24T06:17:14.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.603+0000 7f7a6ef41640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6815a5b0 con 0x7f7a680acd70 2023-11-24T06:17:14.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.603+0000 7f7a6ef41640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6806ce70 con 0x7f7a68070490 2023-11-24T06:17:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.603+0000 7f7a6ccdd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7a680acd70 0x7f7a6806dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.603+0000 7f7a6ccdd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f7a680acd70 0x7f7a6806dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:33710/0 (socket says 172.21.15.5:33710) 2023-11-24T06:17:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.603+0000 7f7a6ccdd640 1 -- 172.21.15.5:0/1201684143 learned_addr learned my addr 172.21.15.5:0/1201684143 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.604+0000 7f7a6d4de640 1 --2- 172.21.15.5:0/1201684143 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a68070490 0x7f7a68070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.604+0000 7f7a6ccdd640 1 -- 172.21.15.5:0/1201684143 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a68070490 msgr2=0x7f7a68070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:14.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.604+0000 7f7a6ccdd640 1 --2- 172.21.15.5:0/1201684143 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a68070490 0x7f7a68070920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.604+0000 7f7a6ccdd640 1 -- 172.21.15.5:0/1201684143 >> v1:172.21.15.6:6789/0 conn(0x7f7a680ae490 legacy=0x7f7a680ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:14.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.604+0000 7f7a6ccdd640 1 -- 172.21.15.5:0/1201684143 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6806d150 con 0x7f7a680acd70 2023-11-24T06:17:14.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.605+0000 7f7a6ccdd640 1 --2- 172.21.15.5:0/1201684143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a680acd70 0x7f7a6806dc30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009870 tx=0x7f7a6402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d55cdb26311336f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:14.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.606+0000 7f7a5f7fe640 1 -- 172.21.15.5:0/1201684143 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64030840 con 0x7f7a680acd70 2023-11-24T06:17:14.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.606+0000 7f7a5f7fe640 1 -- 172.21.15.5:0/1201684143 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a640309a0 con 0x7f7a680acd70 2023-11-24T06:17:14.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.606+0000 7f7a5f7fe640 1 -- 172.21.15.5:0/1201684143 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64030d30 con 0x7f7a680acd70 2023-11-24T06:17:14.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.607+0000 7f7a6ef41640 1 -- 172.21.15.5:0/1201684143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a680acd70 msgr2=0x7f7a6806dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:14.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.607+0000 7f7a6ef41640 1 --2- 172.21.15.5:0/1201684143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a680acd70 0x7f7a6806dc30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7a64009870 tx=0x7f7a6402b980 comp rx=0 tx=0).stop 2023-11-24T06:17:14.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.607+0000 7f7a6ef41640 1 -- 172.21.15.5:0/1201684143 shutdown_connections 2023-11-24T06:17:14.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.607+0000 7f7a6ef41640 1 --2- 172.21.15.5:0/1201684143 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a68070490 0x7f7a68070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.607+0000 7f7a6ef41640 1 --2- 172.21.15.5:0/1201684143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a680acd70 0x7f7a6806dc30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.607+0000 7f7a6ef41640 1 -- 172.21.15.5:0/1201684143 >> 172.21.15.5:0/1201684143 conn(0x7f7a680b0ba0 msgr2=0x7f7a680b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:14.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.607+0000 7f7a6ef41640 1 -- 172.21.15.5:0/1201684143 shutdown_connections 2023-11-24T06:17:14.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.607+0000 7f7a6ef41640 1 -- 172.21.15.5:0/1201684143 wait complete. 2023-11-24T06:17:14.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.609+0000 7f7a6ef41640 1 Processor -- start 2023-11-24T06:17:14.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.609+0000 7f7a6ef41640 1 -- start start 2023-11-24T06:17:14.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.610+0000 7f7a6ef41640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a68070490 0x7f7a681571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:14.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.610+0000 7f7a6ef41640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a680acd70 0x7f7a6815b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.610+0000 7f7a6ef41640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a680ae490 0x7f7a6815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.610+0000 7f7a6ef41640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6814ef20 con 0x7f7a680acd70 2023-11-24T06:17:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.610+0000 7f7a6ef41640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a681599e0 con 0x7f7a68070490 2023-11-24T06:17:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.610+0000 7f7a6ef41640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68159cc0 con 0x7f7a680ae490 2023-11-24T06:17:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.610+0000 7f7a5ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a680acd70 0x7f7a6815b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.610+0000 7f7a5ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a680acd70 0x7f7a6815b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50314/0 (socket says 172.21.15.5:50314) 2023-11-24T06:17:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.610+0000 7f7a5ffff640 1 -- 172.21.15.5:0/2679478671 learned_addr learned my addr 172.21.15.5:0/2679478671 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.610+0000 7f7a6ccdd640 1 --2- 172.21.15.5:0/2679478671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a68070490 0x7f7a681571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.611+0000 7f7a6d4de640 1 --2- 172.21.15.5:0/2679478671 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a680ae490 0x7f7a6815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.611+0000 7f7a5ffff640 1 -- 172.21.15.5:0/2679478671 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a680ae490 msgr2=0x7f7a6815b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.611+0000 7f7a5ffff640 1 --2- 172.21.15.5:0/2679478671 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a680ae490 0x7f7a6815b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.611+0000 7f7a5ffff640 1 -- 172.21.15.5:0/2679478671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a68070490 msgr2=0x7f7a681571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.611+0000 7f7a5ffff640 1 --2- 172.21.15.5:0/2679478671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a68070490 0x7f7a681571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.611+0000 7f7a5ffff640 1 -- 172.21.15.5:0/2679478671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a640094d0 con 0x7f7a680acd70 2023-11-24T06:17:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.611+0000 7f7a5ffff640 1 --2- 172.21.15.5:0/2679478671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a680acd70 0x7f7a6815b060 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800c3c0 tx=0x7f7a5800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.611+0000 7f7a5d7fa640 1 -- 172.21.15.5:0/2679478671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a580052d0 con 0x7f7a680acd70 2023-11-24T06:17:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.611+0000 7f7a5d7fa640 1 -- 172.21.15.5:0/2679478671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a58005450 con 0x7f7a680acd70 2023-11-24T06:17:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.611+0000 7f7a5d7fa640 1 -- 172.21.15.5:0/2679478671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a58007260 con 0x7f7a680acd70 2023-11-24T06:17:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.612+0000 7f7a6ef41640 1 -- 172.21.15.5:0/2679478671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a68157bd0 con 0x7f7a680acd70 2023-11-24T06:17:14.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.612+0000 7f7a6ef41640 1 -- 172.21.15.5:0/2679478671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a681576e0 con 0x7f7a680acd70 2023-11-24T06:17:14.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.613+0000 7f7a6ef41640 1 -- 172.21.15.5:0/2679478671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a681580f0 con 0x7f7a680acd70 2023-11-24T06:17:14.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.614+0000 7f7a5d7fa640 1 -- 172.21.15.5:0/2679478671 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a580073c0 con 0x7f7a680acd70 2023-11-24T06:17:14.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.615+0000 7f7a5d7fa640 1 --2- 172.21.15.5:0/2679478671 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7a5406ae40 0x7f7a5406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:14.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.615+0000 7f7a5d7fa640 1 -- 172.21.15.5:0/2679478671 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7a58088670 con 0x7f7a680acd70 2023-11-24T06:17:14.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.615+0000 7f7a6ccdd640 1 --2- 172.21.15.5:0/2679478671 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7a5406ae40 0x7f7a5406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:14.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.616+0000 7f7a6ccdd640 1 --2- 172.21.15.5:0/2679478671 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7a5406ae40 0x7f7a5406d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7a64036000 tx=0x7f7a64008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.619+0000 7f7a5d7fa640 1 -- 172.21.15.5:0/2679478671 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a681580f0 con 0x7f7a680acd70 2023-11-24T06:17:14.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:14.812+0000 7f7a6ef41640 1 -- 172.21.15.5:0/2679478671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7a681580f0 con 0x7f7a680acd70 2023-11-24T06:17:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.372+0000 7f7a5d7fa640 1 -- 172.21.15.5:0/2679478671 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7a681580f0 con 0x7f7a680acd70 2023-11-24T06:17:15.374 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2023-11-24T06:17:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.376+0000 7f7a6ef41640 1 -- 172.21.15.5:0/2679478671 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7a5406ae40 msgr2=0x7f7a5406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.376+0000 7f7a6ef41640 1 --2- 172.21.15.5:0/2679478671 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7a5406ae40 0x7f7a5406d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7a64036000 tx=0x7f7a64008030 comp rx=0 tx=0).stop 2023-11-24T06:17:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.376+0000 7f7a6ef41640 1 -- 172.21.15.5:0/2679478671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a680acd70 msgr2=0x7f7a6815b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.376+0000 7f7a6ef41640 1 --2- 172.21.15.5:0/2679478671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a680acd70 0x7f7a6815b060 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800c3c0 tx=0x7f7a5800c880 comp rx=0 tx=0).stop 2023-11-24T06:17:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.376+0000 7f7a6ef41640 1 -- 172.21.15.5:0/2679478671 shutdown_connections 2023-11-24T06:17:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.376+0000 7f7a6ef41640 1 --2- 172.21.15.5:0/2679478671 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7a5406ae40 0x7f7a5406d300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.376+0000 7f7a6ef41640 1 --2- 172.21.15.5:0/2679478671 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7a680ae490 0x7f7a6815b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.376+0000 7f7a6ef41640 1 --2- 172.21.15.5:0/2679478671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a680acd70 0x7f7a6815b060 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.376+0000 7f7a6ef41640 1 --2- 172.21.15.5:0/2679478671 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7a68070490 0x7f7a681571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.376+0000 7f7a6ef41640 1 -- 172.21.15.5:0/2679478671 >> 172.21.15.5:0/2679478671 conn(0x7f7a680b0ba0 msgr2=0x7f7a680aee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.376+0000 7f7a6ef41640 1 -- 172.21.15.5:0/2679478671 shutdown_connections 2023-11-24T06:17:15.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.377+0000 7f7a6ef41640 1 -- 172.21.15.5:0/2679478671 wait complete. 2023-11-24T06:17:15.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T06:17:15.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.627+0000 7f0e3cfdb640 1 Processor -- start 2023-11-24T06:17:15.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.627+0000 7f0e3cfdb640 1 -- start start 2023-11-24T06:17:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.628+0000 7f0e3cfdb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e38144770 0x7f0e3815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.628+0000 7f0e3cfdb640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0e380b35a0 0x7f0e3815f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.629+0000 7f0e3cfdb640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0e3806c9d0 0x7f0e3806ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.629+0000 7f0e3cfdb640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e380b43b0 con 0x7f0e380b35a0 2023-11-24T06:17:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.629+0000 7f0e3cfdb640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e3815f930 con 0x7f0e3806c9d0 2023-11-24T06:17:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.629+0000 7f0e3cfdb640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380b3b10 con 0x7f0e38144770 2023-11-24T06:17:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.629+0000 7f0e36ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0e380b35a0 0x7f0e3815f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.629+0000 7f0e36ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0e380b35a0 0x7f0e3815f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50320/0 (socket says 172.21.15.5:50320) 2023-11-24T06:17:15.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.629+0000 7f0e36ffd640 1 -- 172.21.15.5:0/4197692041 learned_addr learned my addr 172.21.15.5:0/4197692041 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:15.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.629+0000 7f0e377fe640 1 --2- 172.21.15.5:0/4197692041 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e38144770 0x7f0e3815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:15.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.629+0000 7f0e37fff640 1 --2- 172.21.15.5:0/4197692041 >> v2:172.21.15.6:3300/0 conn(0x7f0e3806c9d0 0x7f0e3806ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:15.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.629+0000 7f0e36ffd640 1 -- 172.21.15.5:0/4197692041 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e38144770 msgr2=0x7f0e3815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:15.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.629+0000 7f0e36ffd640 1 --2- 172.21.15.5:0/4197692041 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e38144770 0x7f0e3815eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:15.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.630+0000 7f0e36ffd640 1 -- 172.21.15.5:0/4197692041 >> v2:172.21.15.6:3300/0 conn(0x7f0e3806c9d0 msgr2=0x7f0e3806ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:15.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.630+0000 7f0e36ffd640 1 --2- 172.21.15.5:0/4197692041 >> v2:172.21.15.6:3300/0 conn(0x7f0e3806c9d0 0x7f0e3806ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:15.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.630+0000 7f0e36ffd640 1 -- 172.21.15.5:0/4197692041 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e380b3df0 con 0x7f0e380b35a0 2023-11-24T06:17:15.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.631+0000 7f0e36ffd640 1 --2- 172.21.15.5:0/4197692041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e380b35a0 0x7f0e3815f0a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009e50 tx=0x7f0e2002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e8ab508eed061a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:15.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.631+0000 7f0e367fc640 1 -- 172.21.15.5:0/4197692041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2002ca70 con 0x7f0e380b35a0 2023-11-24T06:17:15.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.631+0000 7f0e367fc640 1 -- 172.21.15.5:0/4197692041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2002cbd0 con 0x7f0e380b35a0 2023-11-24T06:17:15.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.631+0000 7f0e367fc640 1 -- 172.21.15.5:0/4197692041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e200343f0 con 0x7f0e380b35a0 2023-11-24T06:17:15.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.632+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/4197692041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e380b35a0 msgr2=0x7f0e3815f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:15.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.632+0000 7f0e3cfdb640 1 --2- 172.21.15.5:0/4197692041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e380b35a0 0x7f0e3815f0a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0e20009e50 tx=0x7f0e2002bed0 comp rx=0 tx=0).stop 2023-11-24T06:17:15.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.632+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/4197692041 shutdown_connections 2023-11-24T06:17:15.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.632+0000 7f0e3cfdb640 1 --2- 172.21.15.5:0/4197692041 >> v2:172.21.15.6:3300/0 conn(0x7f0e3806c9d0 0x7f0e3806ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:15.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.632+0000 7f0e3cfdb640 1 --2- 172.21.15.5:0/4197692041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e380b35a0 0x7f0e3815f0a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:15.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.632+0000 7f0e3cfdb640 1 --2- 172.21.15.5:0/4197692041 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e38144770 0x7f0e3815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:15.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.632+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/4197692041 >> 172.21.15.5:0/4197692041 conn(0x7f0e380cec10 msgr2=0x7f0e380cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:15.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.632+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/4197692041 shutdown_connections 2023-11-24T06:17:15.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.632+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/4197692041 wait complete. 2023-11-24T06:17:15.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.634+0000 7f0e3cfdb640 1 Processor -- start 2023-11-24T06:17:15.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.634+0000 7f0e3cfdb640 1 -- start start 2023-11-24T06:17:15.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.635+0000 7f0e3cfdb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e3806c9d0 0x7f0e380cb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:15.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.635+0000 7f0e3cfdb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e380b35a0 0x7f0e380afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:15.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.635+0000 7f0e3cfdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e38144770 0x7f0e380ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:15.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.635+0000 7f0e3cfdb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38141710 con 0x7f0e38144770 2023-11-24T06:17:15.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.635+0000 7f0e3cfdb640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38142e00 con 0x7f0e3806c9d0 2023-11-24T06:17:15.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.635+0000 7f0e3cfdb640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e381430e0 con 0x7f0e380b35a0 2023-11-24T06:17:15.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.635+0000 7f0e37fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e38144770 0x7f0e380ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:15.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.635+0000 7f0e37fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e38144770 0x7f0e380ac870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50322/0 (socket says 172.21.15.5:50322) 2023-11-24T06:17:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.635+0000 7f0e37fff640 1 -- 172.21.15.5:0/2298857731 learned_addr learned my addr 172.21.15.5:0/2298857731 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.635+0000 7f0e377fe640 1 --2- 172.21.15.5:0/2298857731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e3806c9d0 0x7f0e380cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.636+0000 7f0e37fff640 1 -- 172.21.15.5:0/2298857731 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e380b35a0 msgr2=0x7f0e380afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.636+0000 7f0e37fff640 1 --2- 172.21.15.5:0/2298857731 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e380b35a0 0x7f0e380afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.636+0000 7f0e37fff640 1 -- 172.21.15.5:0/2298857731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e3806c9d0 msgr2=0x7f0e380cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.636+0000 7f0e37fff640 1 --2- 172.21.15.5:0/2298857731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e3806c9d0 0x7f0e380cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.636+0000 7f0e37fff640 1 -- 172.21.15.5:0/2298857731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e20009b30 con 0x7f0e38144770 2023-11-24T06:17:15.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.636+0000 7f0e37fff640 1 --2- 172.21.15.5:0/2298857731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e38144770 0x7f0e380ac870 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000d800 tx=0x7f0e3000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.636+0000 7f0e2ffff640 1 -- 172.21.15.5:0/2298857731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3000b6c0 con 0x7f0e38144770 2023-11-24T06:17:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.636+0000 7f0e2ffff640 1 -- 172.21.15.5:0/2298857731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3000b840 con 0x7f0e38144770 2023-11-24T06:17:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.636+0000 7f0e2ffff640 1 -- 172.21.15.5:0/2298857731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3000bbf0 con 0x7f0e38144770 2023-11-24T06:17:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.637+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/2298857731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e381433c0 con 0x7f0e38144770 2023-11-24T06:17:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.637+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/2298857731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e380cbaa0 con 0x7f0e38144770 2023-11-24T06:17:15.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.638+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/2298857731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e38141470 con 0x7f0e38144770 2023-11-24T06:17:15.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.639+0000 7f0e2ffff640 1 -- 172.21.15.5:0/2298857731 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0e3000bd50 con 0x7f0e38144770 2023-11-24T06:17:15.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.640+0000 7f0e2ffff640 1 --2- 172.21.15.5:0/2298857731 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0e1806ae40 0x7f0e1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:15.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.640+0000 7f0e2ffff640 1 -- 172.21.15.5:0/2298857731 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f0e300899d0 con 0x7f0e38144770 2023-11-24T06:17:15.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.640+0000 7f0e377fe640 1 --2- 172.21.15.5:0/2298857731 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0e1806ae40 0x7f0e1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:15.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.641+0000 7f0e377fe640 1 --2- 172.21.15.5:0/2298857731 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0e1806ae40 0x7f0e1806d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e28006fd0 tx=0x7f0e28008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:15.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.645+0000 7f0e2ffff640 1 -- 172.21.15.5:0/2298857731 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e38141470 con 0x7f0e38144770 2023-11-24T06:17:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:15.840+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/2298857731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0e38141850 con 0x7f0e38144770 2023-11-24T06:17:16.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.389+0000 7f0e2ffff640 1 -- 172.21.15.5:0/2298857731 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0e38141850 con 0x7f0e38144770 2023-11-24T06:17:16.392 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2023-11-24T06:17:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.393+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/2298857731 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0e1806ae40 msgr2=0x7f0e1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.393+0000 7f0e3cfdb640 1 --2- 172.21.15.5:0/2298857731 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0e1806ae40 0x7f0e1806d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e28006fd0 tx=0x7f0e28008040 comp rx=0 tx=0).stop 2023-11-24T06:17:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.393+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/2298857731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e38144770 msgr2=0x7f0e380ac870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.393+0000 7f0e3cfdb640 1 --2- 172.21.15.5:0/2298857731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e38144770 0x7f0e380ac870 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000d800 tx=0x7f0e3000dcc0 comp rx=0 tx=0).stop 2023-11-24T06:17:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.393+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/2298857731 shutdown_connections 2023-11-24T06:17:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.393+0000 7f0e3cfdb640 1 --2- 172.21.15.5:0/2298857731 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0e1806ae40 0x7f0e1806d300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.393+0000 7f0e3cfdb640 1 --2- 172.21.15.5:0/2298857731 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e380b35a0 0x7f0e380afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.393+0000 7f0e3cfdb640 1 --2- 172.21.15.5:0/2298857731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e38144770 0x7f0e380ac870 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.393+0000 7f0e3cfdb640 1 --2- 172.21.15.5:0/2298857731 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e3806c9d0 0x7f0e380cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.393+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/2298857731 >> 172.21.15.5:0/2298857731 conn(0x7f0e380cec10 msgr2=0x7f0e38071780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.394+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/2298857731 shutdown_connections 2023-11-24T06:17:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.395+0000 7f0e3cfdb640 1 -- 172.21.15.5:0/2298857731 wait complete. 2023-11-24T06:17:16.430 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T06:17:16.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.619+0000 7fa37769b640 1 Processor -- start 2023-11-24T06:17:16.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.619+0000 7fa37769b640 1 -- start start 2023-11-24T06:17:16.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.621+0000 7fa37769b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa3700cd150 0x7fa37006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:16.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.621+0000 7fa37769b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3700702a0 0x7fa37006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:16.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.621+0000 7fa37769b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa370142d50 con 0x7fa37006ec60 2023-11-24T06:17:16.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.621+0000 7fa37769b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3700cd530 con 0x7fa3700cd150 2023-11-24T06:17:16.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.621+0000 7fa37769b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700706c0 con 0x7fa3700702a0 2023-11-24T06:17:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.621+0000 7fa375c38640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa37006ec60 0x7fa370160fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50826/0 (socket says 172.21.15.5:50826) 2023-11-24T06:17:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.621+0000 7fa375c38640 1 -- 172.21.15.5:0/666147219 learned_addr learned my addr 172.21.15.5:0/666147219 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.622+0000 7fa374c36640 1 --2- 172.21.15.5:0/666147219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3700702a0 0x7fa37006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.622+0000 7fa375437640 1 --2- 172.21.15.5:0/666147219 >> v2:172.21.15.6:3300/0 conn(0x7fa3700cd150 0x7fa37006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.622+0000 7fa36ffff640 1 -- 172.21.15.5:0/666147219 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901105043 0 0) 0x7fa370142d50 con 0x7fa37006ec60 2023-11-24T06:17:16.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.622+0000 7fa36ffff640 1 -- 172.21.15.5:0/666147219 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa358003520 con 0x7fa37006ec60 2023-11-24T06:17:16.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.622+0000 7fa36ffff640 1 -- 172.21.15.5:0/666147219 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2698517472 0 0) 0x7fa358003520 con 0x7fa37006ec60 2023-11-24T06:17:16.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.622+0000 7fa36ffff640 1 -- 172.21.15.5:0/666147219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3700702a0 msgr2=0x7fa37006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.622+0000 7fa36ffff640 1 --2- 172.21.15.5:0/666147219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3700702a0 0x7fa37006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.622+0000 7fa36ffff640 1 -- 172.21.15.5:0/666147219 >> v2:172.21.15.6:3300/0 conn(0x7fa3700cd150 msgr2=0x7fa37006fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.622+0000 7fa36ffff640 1 --2- 172.21.15.5:0/666147219 >> v2:172.21.15.6:3300/0 conn(0x7fa3700cd150 0x7fa37006fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.622+0000 7fa36ffff640 1 -- 172.21.15.5:0/666147219 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37006f430 con 0x7fa37006ec60 2023-11-24T06:17:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.622+0000 7fa36ffff640 1 -- 172.21.15.5:0/666147219 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fa368001670 con 0x7fa37006ec60 2023-11-24T06:17:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa36ffff640 1 -- 172.21.15.5:0/666147219 >> v1:172.21.15.5:6789/0 conn(0x7fa37006ec60 legacy=0x7fa370160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:16.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa36ffff640 1 --2- 172.21.15.5:0/666147219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa358003970 0x7fa358003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:16.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa36ffff640 1 --2- 172.21.15.5:0/666147219 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3580047c0 0x7fa358004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:16.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa36ffff640 1 --2- 172.21.15.5:0/666147219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3700702a0 0x7fa358005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:16.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa36ffff640 1 -- 172.21.15.5:0/666147219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700cd530 con 0x7fa358003970 2023-11-24T06:17:16.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa36ffff640 1 -- 172.21.15.5:0/666147219 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700706c0 con 0x7fa3580047c0 2023-11-24T06:17:16.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa36ffff640 1 -- 172.21.15.5:0/666147219 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35800b980 con 0x7fa3700702a0 2023-11-24T06:17:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa375437640 1 --2- 172.21.15.5:0/666147219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa358003970 0x7fa358003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa375437640 1 -- 172.21.15.5:0/666147219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3700702a0 msgr2=0x7fa358005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa375c38640 1 --2- 172.21.15.5:0/666147219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3700702a0 0x7fa358005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa374c36640 1 --2- 172.21.15.5:0/666147219 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3580047c0 0x7fa358004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa375437640 1 --2- 172.21.15.5:0/666147219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3700702a0 0x7fa358005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa375437640 1 -- 172.21.15.5:0/666147219 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3580047c0 msgr2=0x7fa358004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa375437640 1 --2- 172.21.15.5:0/666147219 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa3580047c0 0x7fa358004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa375437640 1 -- 172.21.15.5:0/666147219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa358003520 con 0x7fa358003970 2023-11-24T06:17:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.623+0000 7fa375c38640 1 --2- 172.21.15.5:0/666147219 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3700702a0 0x7fa358005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.625+0000 7fa375437640 1 --2- 172.21.15.5:0/666147219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa358003970 0x7fa358003d90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa36000e170 tx=0x7fa3580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff8e8dc3d1fabe1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.625+0000 7fa36ffff640 1 -- 172.21.15.5:0/666147219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36002cc20 con 0x7fa358003970 2023-11-24T06:17:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.625+0000 7fa36ffff640 1 -- 172.21.15.5:0/666147219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36000f040 con 0x7fa358003970 2023-11-24T06:17:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.625+0000 7fa36ffff640 1 -- 172.21.15.5:0/666147219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360034440 con 0x7fa358003970 2023-11-24T06:17:16.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.625+0000 7fa37769b640 1 -- 172.21.15.5:0/666147219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa358003970 msgr2=0x7fa358003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:16.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.625+0000 7fa37769b640 1 --2- 172.21.15.5:0/666147219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa358003970 0x7fa358003d90 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa36000e170 tx=0x7fa3580057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:16.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.625+0000 7fa375c38640 1 -- 172.21.15.5:0/666147219 reap_dead start 2023-11-24T06:17:16.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.626+0000 7fa37769b640 1 -- 172.21.15.5:0/666147219 shutdown_connections 2023-11-24T06:17:16.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.626+0000 7fa37769b640 1 -- 172.21.15.5:0/666147219 >> 172.21.15.5:0/666147219 conn(0x7fa3700cdb20 msgr2=0x7fa3700cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:16.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.626+0000 7fa37769b640 1 -- 172.21.15.5:0/666147219 shutdown_connections 2023-11-24T06:17:16.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.626+0000 7fa37769b640 1 -- 172.21.15.5:0/666147219 wait complete. 2023-11-24T06:17:16.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.628+0000 7fa37769b640 1 Processor -- start 2023-11-24T06:17:16.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.628+0000 7fa37769b640 1 -- start start 2023-11-24T06:17:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.629+0000 7fa37769b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa358003970 0x7fa3700b21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.629+0000 7fa37769b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3701f3380 0x7fa370144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.629+0000 7fa37769b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3701450e0 0x7fa3700ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.629+0000 7fa37769b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa370145660 con 0x7fa3701450e0 2023-11-24T06:17:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.629+0000 7fa37769b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa370145940 con 0x7fa358003970 2023-11-24T06:17:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.629+0000 7fa37769b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa370145c20 con 0x7fa3701f3380 2023-11-24T06:17:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.629+0000 7fa375c38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3701450e0 0x7fa3700ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.629+0000 7fa375c38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3701450e0 0x7fa3700ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50334/0 (socket says 172.21.15.5:50334) 2023-11-24T06:17:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.629+0000 7fa375c38640 1 -- 172.21.15.5:0/3238809998 learned_addr learned my addr 172.21.15.5:0/3238809998 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.629+0000 7fa375437640 1 --2- 172.21.15.5:0/3238809998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa358003970 0x7fa3700b21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.629+0000 7fa374c36640 1 --2- 172.21.15.5:0/3238809998 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3701f3380 0x7fa370144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.630+0000 7fa375c38640 1 -- 172.21.15.5:0/3238809998 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3701f3380 msgr2=0x7fa370144ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.630+0000 7fa375c38640 1 --2- 172.21.15.5:0/3238809998 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3701f3380 0x7fa370144ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.630+0000 7fa375c38640 1 -- 172.21.15.5:0/3238809998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa358003970 msgr2=0x7fa3700b21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.630+0000 7fa375c38640 1 --2- 172.21.15.5:0/3238809998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa358003970 0x7fa3700b21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.630+0000 7fa375c38640 1 -- 172.21.15.5:0/3238809998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa370145f00 con 0x7fa3701450e0 2023-11-24T06:17:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.630+0000 7fa375c38640 1 --2- 172.21.15.5:0/3238809998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3701450e0 0x7fa3700ac880 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa370072c60 tx=0x7fa36800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.630+0000 7fa36dffb640 1 -- 172.21.15.5:0/3238809998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368007230 con 0x7fa3701450e0 2023-11-24T06:17:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.630+0000 7fa36dffb640 1 -- 172.21.15.5:0/3238809998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa368007390 con 0x7fa3701450e0 2023-11-24T06:17:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.630+0000 7fa36dffb640 1 -- 172.21.15.5:0/3238809998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3680047e0 con 0x7fa3701450e0 2023-11-24T06:17:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.630+0000 7fa37769b640 1 -- 172.21.15.5:0/3238809998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3701461e0 con 0x7fa3701450e0 2023-11-24T06:17:16.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.630+0000 7fa37769b640 1 -- 172.21.15.5:0/3238809998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3700b2710 con 0x7fa3701450e0 2023-11-24T06:17:16.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.632+0000 7fa37769b640 1 -- 172.21.15.5:0/3238809998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37006fb50 con 0x7fa3701450e0 2023-11-24T06:17:16.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.633+0000 7fa36dffb640 1 -- 172.21.15.5:0/3238809998 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa368004940 con 0x7fa3701450e0 2023-11-24T06:17:16.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.633+0000 7fa36dffb640 1 --2- 172.21.15.5:0/3238809998 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa35c06ae40 0x7fa35c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:16.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.634+0000 7fa36dffb640 1 -- 172.21.15.5:0/3238809998 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fa368089380 con 0x7fa3701450e0 2023-11-24T06:17:16.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.634+0000 7fa375437640 1 --2- 172.21.15.5:0/3238809998 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa35c06ae40 0x7fa35c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:16.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.634+0000 7fa375437640 1 --2- 172.21.15.5:0/3238809998 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa35c06ae40 0x7fa35c06d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa360036000 tx=0x7fa36002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:16.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.638+0000 7fa36dffb640 1 -- 172.21.15.5:0/3238809998 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3680543f0 con 0x7fa3701450e0 2023-11-24T06:17:16.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:16.872+0000 7fa37769b640 1 -- 172.21.15.5:0/3238809998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa37004e920 con 0x7fa3701450e0 2023-11-24T06:17:17.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:17.809+0000 7fa36dffb640 1 -- 172.21.15.5:0/3238809998 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa368057a10 con 0x7fa3701450e0 2023-11-24T06:17:17.813 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 4 and data pool 5 2023-11-24T06:17:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:17.813+0000 7fa37769b640 1 -- 172.21.15.5:0/3238809998 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa35c06ae40 msgr2=0x7fa35c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:17.813+0000 7fa37769b640 1 --2- 172.21.15.5:0/3238809998 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa35c06ae40 0x7fa35c06d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa360036000 tx=0x7fa36002e000 comp rx=0 tx=0).stop 2023-11-24T06:17:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:17.813+0000 7fa37769b640 1 -- 172.21.15.5:0/3238809998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3701450e0 msgr2=0x7fa3700ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:17.813+0000 7fa37769b640 1 --2- 172.21.15.5:0/3238809998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3701450e0 0x7fa3700ac880 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa370072c60 tx=0x7fa36800dc00 comp rx=0 tx=0).stop 2023-11-24T06:17:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:17.813+0000 7fa37769b640 1 -- 172.21.15.5:0/3238809998 shutdown_connections 2023-11-24T06:17:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:17.813+0000 7fa37769b640 1 --2- 172.21.15.5:0/3238809998 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa35c06ae40 0x7fa35c06d300 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:17.813+0000 7fa37769b640 1 --2- 172.21.15.5:0/3238809998 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa3701f3380 0x7fa370144ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:17.813+0000 7fa37769b640 1 --2- 172.21.15.5:0/3238809998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3701450e0 0x7fa3700ac880 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:17.813+0000 7fa37769b640 1 --2- 172.21.15.5:0/3238809998 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa358003970 0x7fa3700b21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:17.813+0000 7fa37769b640 1 -- 172.21.15.5:0/3238809998 >> 172.21.15.5:0/3238809998 conn(0x7fa3700cdb20 msgr2=0x7fa3700d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:17.813+0000 7fa37769b640 1 -- 172.21.15.5:0/3238809998 shutdown_connections 2023-11-24T06:17:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:17.813+0000 7fa37769b640 1 -- 172.21.15.5:0/3238809998 wait complete. 2023-11-24T06:17:17.842 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:17:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.035+0000 7f23cf3ec640 1 Processor -- start 2023-11-24T06:17:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.035+0000 7f23cf3ec640 1 -- start start 2023-11-24T06:17:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.037+0000 7f23cf3ec640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f23c80719c0 0x7f23c80ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.037+0000 7f23cf3ec640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c80ad510 con 0x7f23c8161930 2023-11-24T06:17:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.037+0000 7f23cf3ec640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c80acc50 con 0x7f23c806d520 2023-11-24T06:17:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.037+0000 7f23cf3ec640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c8071e60 con 0x7f23c80719c0 2023-11-24T06:17:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.037+0000 7f23cc987640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f23c80719c0 0x7f23c80ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.038+0000 7f23cc987640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f23c80719c0 0x7f23c80ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50336/0 (socket says 172.21.15.5:50336) 2023-11-24T06:17:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.038+0000 7f23cc987640 1 -- 172.21.15.5:0/3110527840 learned_addr learned my addr 172.21.15.5:0/3110527840 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.038+0000 7f23cd188640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f23c8161930 0x7f23c80ad9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50836/0 (socket says 172.21.15.5:50836) 2023-11-24T06:17:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.038+0000 7f23cc987640 1 -- 172.21.15.5:0/3110527840 >> v1:172.21.15.6:6789/0 conn(0x7f23c806d520 legacy=0x7f23c806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.038+0000 7f23cc987640 1 -- 172.21.15.5:0/3110527840 >> v1:172.21.15.5:6789/0 conn(0x7f23c8161930 legacy=0x7f23c80ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.038+0000 7f23cc987640 1 -- 172.21.15.5:0/3110527840 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c80acec0 con 0x7f23c80719c0 2023-11-24T06:17:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.039+0000 7f23cc987640 1 --2- 172.21.15.5:0/3110527840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23c80719c0 0x7f23c80ae6a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009870 tx=0x7f23b802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876b3b410453ba3d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.039+0000 7f23bffff640 1 -- 172.21.15.5:0/3110527840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8030860 con 0x7f23c80719c0 2023-11-24T06:17:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.039+0000 7f23bffff640 1 -- 172.21.15.5:0/3110527840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b80309c0 con 0x7f23c80719c0 2023-11-24T06:17:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.039+0000 7f23bffff640 1 -- 172.21.15.5:0/3110527840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8030d50 con 0x7f23c80719c0 2023-11-24T06:17:18.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.040+0000 7f23cf3ec640 1 -- 172.21.15.5:0/3110527840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23c80719c0 msgr2=0x7f23c80ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.040+0000 7f23cf3ec640 1 --2- 172.21.15.5:0/3110527840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23c80719c0 0x7f23c80ae6a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009870 tx=0x7f23b802b9a0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.040+0000 7f23cf3ec640 1 -- 172.21.15.5:0/3110527840 shutdown_connections 2023-11-24T06:17:18.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.040+0000 7f23cf3ec640 1 --2- 172.21.15.5:0/3110527840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23c80719c0 0x7f23c80ae6a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.040+0000 7f23cf3ec640 1 -- 172.21.15.5:0/3110527840 >> 172.21.15.5:0/3110527840 conn(0x7f23c80b0d80 msgr2=0x7f23c80b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:18.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.040+0000 7f23cf3ec640 1 -- 172.21.15.5:0/3110527840 shutdown_connections 2023-11-24T06:17:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.040+0000 7f23cf3ec640 1 -- 172.21.15.5:0/3110527840 wait complete. 2023-11-24T06:17:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.042+0000 7f23cf3ec640 1 Processor -- start 2023-11-24T06:17:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.042+0000 7f23cf3ec640 1 -- start start 2023-11-24T06:17:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.043+0000 7f23cf3ec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23c806d520 0x7f23c815e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.043+0000 7f23cf3ec640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23c80719c0 0x7f23c81501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.043+0000 7f23cf3ec640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23c8161930 0x7f23c8099170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.043+0000 7f23cf3ec640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c80ccd20 con 0x7f23c806d520 2023-11-24T06:17:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.043+0000 7f23cf3ec640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c80cd000 con 0x7f23c8161930 2023-11-24T06:17:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.043+0000 7f23cf3ec640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c80cd2e0 con 0x7f23c80719c0 2023-11-24T06:17:18.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.043+0000 7f23cd188640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23c806d520 0x7f23c815e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.043+0000 7f23cd188640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23c806d520 0x7f23c815e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50342/0 (socket says 172.21.15.5:50342) 2023-11-24T06:17:18.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.043+0000 7f23cd188640 1 -- 172.21.15.5:0/919471373 learned_addr learned my addr 172.21.15.5:0/919471373 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.044+0000 7f23cc987640 1 --2- 172.21.15.5:0/919471373 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23c80719c0 0x7f23c81501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.044+0000 7f23cd188640 1 -- 172.21.15.5:0/919471373 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23c80719c0 msgr2=0x7f23c81501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.044+0000 7f23cd989640 1 --2- 172.21.15.5:0/919471373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23c8161930 0x7f23c8099170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.044+0000 7f23cd188640 1 --2- 172.21.15.5:0/919471373 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23c80719c0 0x7f23c81501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.044+0000 7f23cd188640 1 -- 172.21.15.5:0/919471373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23c8161930 msgr2=0x7f23c8099170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.044+0000 7f23cd188640 1 --2- 172.21.15.5:0/919471373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23c8161930 0x7f23c8099170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.044+0000 7f23cd188640 1 -- 172.21.15.5:0/919471373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b80094d0 con 0x7f23c806d520 2023-11-24T06:17:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.044+0000 7f23cd188640 1 --2- 172.21.15.5:0/919471373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23c806d520 0x7f23c815e500 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f23c400c3e0 tx=0x7f23c400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.044+0000 7f23bdffb640 1 -- 172.21.15.5:0/919471373 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40074e0 con 0x7f23c806d520 2023-11-24T06:17:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.044+0000 7f23bdffb640 1 -- 172.21.15.5:0/919471373 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c4007640 con 0x7f23c806d520 2023-11-24T06:17:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.044+0000 7f23bdffb640 1 -- 172.21.15.5:0/919471373 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40049e0 con 0x7f23c806d520 2023-11-24T06:17:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.044+0000 7f23cf3ec640 1 -- 172.21.15.5:0/919471373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c80cd5c0 con 0x7f23c806d520 2023-11-24T06:17:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.044+0000 7f23cf3ec640 1 -- 172.21.15.5:0/919471373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c815ea40 con 0x7f23c806d520 2023-11-24T06:17:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.046+0000 7f23cf3ec640 1 -- 172.21.15.5:0/919471373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c815ebd0 con 0x7f23c806d520 2023-11-24T06:17:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.047+0000 7f23bdffb640 1 -- 172.21.15.5:0/919471373 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f23c4004b40 con 0x7f23c806d520 2023-11-24T06:17:18.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.048+0000 7f23bdffb640 1 --2- 172.21.15.5:0/919471373 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f23ac06ae40 0x7f23ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.048+0000 7f23bdffb640 1 -- 172.21.15.5:0/919471373 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23c40890a0 con 0x7f23c806d520 2023-11-24T06:17:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.048+0000 7f23cc987640 1 --2- 172.21.15.5:0/919471373 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f23ac06ae40 0x7f23ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.049+0000 7f23cc987640 1 --2- 172.21.15.5:0/919471373 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f23ac06ae40 0x7f23ac06d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f23b8036000 tx=0x7f23b803b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:18.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.053+0000 7f23bdffb640 1 -- 172.21.15.5:0/919471373 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23c815ebd0 con 0x7f23c806d520 2023-11-24T06:17:18.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.250+0000 7f23cf3ec640 1 -- 172.21.15.5:0/919471373 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23c80cdae0 con 0x7f23c806d520 2023-11-24T06:17:18.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.252+0000 7f23bdffb640 1 -- 172.21.15.5:0/919471373 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16438 (secure 0 0 0) 0x7f23c80cdae0 con 0x7f23c806d520 2023-11-24T06:17:18.255 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:18.255 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:17:16.874118+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:17:14.868917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:17:15.895189+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:17:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.255+0000 7f23cf3ec640 1 -- 172.21.15.5:0/919471373 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f23ac06ae40 msgr2=0x7f23ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.255+0000 7f23cf3ec640 1 --2- 172.21.15.5:0/919471373 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f23ac06ae40 0x7f23ac06d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f23b8036000 tx=0x7f23b803b040 comp rx=0 tx=0).stop 2023-11-24T06:17:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.255+0000 7f23cf3ec640 1 -- 172.21.15.5:0/919471373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23c806d520 msgr2=0x7f23c815e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.255+0000 7f23cf3ec640 1 --2- 172.21.15.5:0/919471373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23c806d520 0x7f23c815e500 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f23c400c3e0 tx=0x7f23c400c8a0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.255+0000 7f23cf3ec640 1 -- 172.21.15.5:0/919471373 shutdown_connections 2023-11-24T06:17:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.255+0000 7f23cf3ec640 1 --2- 172.21.15.5:0/919471373 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f23ac06ae40 0x7f23ac06d300 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.255+0000 7f23cf3ec640 1 --2- 172.21.15.5:0/919471373 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f23c80719c0 0x7f23c81501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.255+0000 7f23cf3ec640 1 --2- 172.21.15.5:0/919471373 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f23c8161930 0x7f23c8099170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.255+0000 7f23cf3ec640 1 --2- 172.21.15.5:0/919471373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23c806d520 0x7f23c815e500 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.255+0000 7f23cf3ec640 1 -- 172.21.15.5:0/919471373 >> 172.21.15.5:0/919471373 conn(0x7f23c80b0d80 msgr2=0x7f23c806d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:18.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.255+0000 7f23cf3ec640 1 -- 172.21.15.5:0/919471373 shutdown_connections 2023-11-24T06:17:18.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.256+0000 7f23cf3ec640 1 -- 172.21.15.5:0/919471373 wait complete. 2023-11-24T06:17:18.278 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:17:18.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.502+0000 7fb395c7e640 1 Processor -- start 2023-11-24T06:17:18.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.502+0000 7fb395c7e640 1 -- start start 2023-11-24T06:17:18.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.503+0000 7fb395c7e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3900cf760 0x7fb39015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.503+0000 7fb395c7e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb390071700 con 0x7fb390143570 2023-11-24T06:17:18.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.504+0000 7fb395c7e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb39015b410 con 0x7fb390143950 2023-11-24T06:17:18.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.504+0000 7fb395c7e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39015b130 con 0x7fb3900cf760 2023-11-24T06:17:18.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.504+0000 7fb394c7c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb390143570 0x7fb3900cf050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50842/0 (socket says 172.21.15.5:50842) 2023-11-24T06:17:18.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.504+0000 7fb394c7c640 1 -- 172.21.15.5:0/907024374 learned_addr learned my addr 172.21.15.5:0/907024374 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.504+0000 7fb38ffff640 1 --2- 172.21.15.5:0/907024374 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3900cf760 0x7fb39015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.504+0000 7fb38f7fe640 1 -- 172.21.15.5:0/907024374 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 867354807 0 0) 0x7fb380001490 con 0x7fb390143570 2023-11-24T06:17:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.504+0000 7fb38f7fe640 1 -- 172.21.15.5:0/907024374 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb378003520 con 0x7fb390143570 2023-11-24T06:17:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.505+0000 7fb38f7fe640 1 -- 172.21.15.5:0/907024374 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2678920375 0 0) 0x7fb378003520 con 0x7fb390143570 2023-11-24T06:17:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.505+0000 7fb38f7fe640 1 -- 172.21.15.5:0/907024374 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3900cf760 msgr2=0x7fb39015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.505+0000 7fb38f7fe640 1 --2- 172.21.15.5:0/907024374 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3900cf760 0x7fb39015a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.505+0000 7fb38f7fe640 1 -- 172.21.15.5:0/907024374 >> v1:172.21.15.6:6789/0 conn(0x7fb390143950 legacy=0x7fb3900b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.505+0000 7fb38f7fe640 1 -- 172.21.15.5:0/907024374 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39015dbd0 con 0x7fb390143570 2023-11-24T06:17:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.505+0000 7fb38f7fe640 1 -- 172.21.15.5:0/907024374 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fb380001b80 con 0x7fb390143570 2023-11-24T06:17:18.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.505+0000 7fb38f7fe640 1 -- 172.21.15.5:0/907024374 >> v1:172.21.15.5:6789/0 conn(0x7fb390143570 legacy=0x7fb3900cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.505+0000 7fb38f7fe640 1 --2- 172.21.15.5:0/907024374 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3900cf760 0x7fb378003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.505+0000 7fb38f7fe640 1 --2- 172.21.15.5:0/907024374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb378003f90 0x7fb3780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.505+0000 7fb38f7fe640 1 --2- 172.21.15.5:0/907024374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378004d00 0x7fb378005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.505+0000 7fb38f7fe640 1 -- 172.21.15.5:0/907024374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39015b130 con 0x7fb378004d00 2023-11-24T06:17:18.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.505+0000 7fb38f7fe640 1 -- 172.21.15.5:0/907024374 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37800dd80 con 0x7fb378003f90 2023-11-24T06:17:18.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.505+0000 7fb38f7fe640 1 -- 172.21.15.5:0/907024374 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37800df40 con 0x7fb3900cf760 2023-11-24T06:17:18.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.505+0000 7fb39547d640 1 --2- 172.21.15.5:0/907024374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378004d00 0x7fb378005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.506+0000 7fb38ffff640 1 --2- 172.21.15.5:0/907024374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb378003f90 0x7fb3780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.506+0000 7fb394c7c640 1 --2- 172.21.15.5:0/907024374 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3900cf760 0x7fb378003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.506+0000 7fb39547d640 1 -- 172.21.15.5:0/907024374 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3900cf760 msgr2=0x7fb378003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:18.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.506+0000 7fb39547d640 1 --2- 172.21.15.5:0/907024374 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb3900cf760 0x7fb378003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.506+0000 7fb39547d640 1 -- 172.21.15.5:0/907024374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb378003f90 msgr2=0x7fb3780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:18.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.506+0000 7fb39547d640 1 --2- 172.21.15.5:0/907024374 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb378003f90 0x7fb3780047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.506+0000 7fb39547d640 1 -- 172.21.15.5:0/907024374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb378003520 con 0x7fb378004d00 2023-11-24T06:17:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.507+0000 7fb39547d640 1 --2- 172.21.15.5:0/907024374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378004d00 0x7fb378005180 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb388001720 tx=0x7fb3780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6dc7ab30e87c917 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.507+0000 7fb38f7fe640 1 -- 172.21.15.5:0/907024374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3880052d0 con 0x7fb378004d00 2023-11-24T06:17:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.507+0000 7fb38f7fe640 1 -- 172.21.15.5:0/907024374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38802e050 con 0x7fb378004d00 2023-11-24T06:17:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.507+0000 7fb38f7fe640 1 -- 172.21.15.5:0/907024374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388003830 con 0x7fb378004d00 2023-11-24T06:17:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.508+0000 7fb395c7e640 1 -- 172.21.15.5:0/907024374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378004d00 msgr2=0x7fb378005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.508+0000 7fb395c7e640 1 --2- 172.21.15.5:0/907024374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378004d00 0x7fb378005180 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb388001720 tx=0x7fb3780057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.508+0000 7fb39547d640 1 -- 172.21.15.5:0/907024374 reap_dead start 2023-11-24T06:17:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.508+0000 7fb395c7e640 1 -- 172.21.15.5:0/907024374 shutdown_connections 2023-11-24T06:17:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.508+0000 7fb395c7e640 1 -- 172.21.15.5:0/907024374 >> 172.21.15.5:0/907024374 conn(0x7fb39006a860 msgr2=0x7fb39006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.508+0000 7fb395c7e640 1 -- 172.21.15.5:0/907024374 shutdown_connections 2023-11-24T06:17:18.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.508+0000 7fb395c7e640 1 -- 172.21.15.5:0/907024374 wait complete. 2023-11-24T06:17:18.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.510+0000 7fb395c7e640 1 Processor -- start 2023-11-24T06:17:18.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.510+0000 7fb395c7e640 1 -- start start 2023-11-24T06:17:18.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.511+0000 7fb395c7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378004d00 0x7fb3900cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.511+0000 7fb395c7e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3901583c0 0x7fb390158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.511+0000 7fb395c7e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb39013c810 0x7fb39013cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.511+0000 7fb395c7e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39013d4d0 con 0x7fb378004d00 2023-11-24T06:17:18.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.511+0000 7fb395c7e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb390158d80 con 0x7fb3901583c0 2023-11-24T06:17:18.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.511+0000 7fb395c7e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb390159d30 con 0x7fb39013c810 2023-11-24T06:17:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.511+0000 7fb394c7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378004d00 0x7fb3900cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.511+0000 7fb394c7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378004d00 0x7fb3900cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50358/0 (socket says 172.21.15.5:50358) 2023-11-24T06:17:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.511+0000 7fb394c7c640 1 -- 172.21.15.5:0/1239080272 learned_addr learned my addr 172.21.15.5:0/1239080272 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.512+0000 7fb394c7c640 1 -- 172.21.15.5:0/1239080272 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb39013c810 msgr2=0x7fb39013cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.512+0000 7fb394c7c640 1 --2- 172.21.15.5:0/1239080272 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb39013c810 0x7fb39013cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.512+0000 7fb394c7c640 1 -- 172.21.15.5:0/1239080272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3901583c0 msgr2=0x7fb390158840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.512+0000 7fb38ffff640 1 --2- 172.21.15.5:0/1239080272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3901583c0 0x7fb390158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.512+0000 7fb394c7c640 1 --2- 172.21.15.5:0/1239080272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3901583c0 0x7fb390158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.512+0000 7fb394c7c640 1 -- 172.21.15.5:0/1239080272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39013a7e0 con 0x7fb378004d00 2023-11-24T06:17:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.512+0000 7fb394c7c640 1 --2- 172.21.15.5:0/1239080272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378004d00 0x7fb3900cd6f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb380005fd0 tx=0x7fb38000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.512+0000 7fb38d7fa640 1 -- 172.21.15.5:0/1239080272 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38000ee50 con 0x7fb378004d00 2023-11-24T06:17:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.512+0000 7fb38d7fa640 1 -- 172.21.15.5:0/1239080272 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb380013420 con 0x7fb378004d00 2023-11-24T06:17:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.512+0000 7fb38d7fa640 1 -- 172.21.15.5:0/1239080272 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3800137d0 con 0x7fb378004d00 2023-11-24T06:17:18.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.512+0000 7fb395c7e640 1 -- 172.21.15.5:0/1239080272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39013aac0 con 0x7fb378004d00 2023-11-24T06:17:18.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.513+0000 7fb395c7e640 1 -- 172.21.15.5:0/1239080272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39013b080 con 0x7fb378004d00 2023-11-24T06:17:18.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.514+0000 7fb395c7e640 1 -- 172.21.15.5:0/1239080272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39004e840 con 0x7fb378004d00 2023-11-24T06:17:18.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.515+0000 7fb38d7fa640 1 -- 172.21.15.5:0/1239080272 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb380013930 con 0x7fb378004d00 2023-11-24T06:17:18.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.516+0000 7fb38d7fa640 1 --2- 172.21.15.5:0/1239080272 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb37c06ae40 0x7fb37c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.516+0000 7fb38d7fa640 1 -- 172.21.15.5:0/1239080272 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb380089c60 con 0x7fb378004d00 2023-11-24T06:17:18.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.516+0000 7fb38ffff640 1 --2- 172.21.15.5:0/1239080272 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb37c06ae40 0x7fb37c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.517+0000 7fb38ffff640 1 --2- 172.21.15.5:0/1239080272 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb37c06ae40 0x7fb37c06d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb3901f2230 tx=0x7fb384008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:18.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.520+0000 7fb38d7fa640 1 -- 172.21.15.5:0/1239080272 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb380054d00 con 0x7fb378004d00 2023-11-24T06:17:18.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.714+0000 7fb395c7e640 1 -- 172.21.15.5:0/1239080272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb390071700 con 0x7fb378004d00 2023-11-24T06:17:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.716+0000 7fb38d7fa640 1 -- 172.21.15.5:0/1239080272 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16425 (secure 0 0 0) 0x7fb390071700 con 0x7fb378004d00 2023-11-24T06:17:18.722 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:18.723 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:17:18.380558+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:17:14.868917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:17:15.895189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:17:18.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.719+0000 7fb395c7e640 1 -- 172.21.15.5:0/1239080272 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb37c06ae40 msgr2=0x7fb37c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.719+0000 7fb395c7e640 1 --2- 172.21.15.5:0/1239080272 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb37c06ae40 0x7fb37c06d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb3901f2230 tx=0x7fb384008040 comp rx=0 tx=0).stop 2023-11-24T06:17:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.719+0000 7fb395c7e640 1 -- 172.21.15.5:0/1239080272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378004d00 msgr2=0x7fb3900cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.719+0000 7fb395c7e640 1 --2- 172.21.15.5:0/1239080272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378004d00 0x7fb3900cd6f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fb380005fd0 tx=0x7fb38000e530 comp rx=0 tx=0).stop 2023-11-24T06:17:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.719+0000 7fb395c7e640 1 -- 172.21.15.5:0/1239080272 shutdown_connections 2023-11-24T06:17:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.719+0000 7fb395c7e640 1 --2- 172.21.15.5:0/1239080272 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb37c06ae40 0x7fb37c06d300 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.719+0000 7fb395c7e640 1 --2- 172.21.15.5:0/1239080272 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb39013c810 0x7fb39013cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.719+0000 7fb395c7e640 1 --2- 172.21.15.5:0/1239080272 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb3901583c0 0x7fb390158840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.719+0000 7fb395c7e640 1 --2- 172.21.15.5:0/1239080272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378004d00 0x7fb3900cd6f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.719+0000 7fb395c7e640 1 -- 172.21.15.5:0/1239080272 >> 172.21.15.5:0/1239080272 conn(0x7fb39006a860 msgr2=0x7fb390141510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.719+0000 7fb395c7e640 1 -- 172.21.15.5:0/1239080272 shutdown_connections 2023-11-24T06:17:18.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.719+0000 7fb395c7e640 1 -- 172.21.15.5:0/1239080272 wait complete. 2023-11-24T06:17:18.740 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T06:17:18.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.968+0000 7f36d6f47640 1 Processor -- start 2023-11-24T06:17:18.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.968+0000 7f36d6f47640 1 -- start start 2023-11-24T06:17:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.970+0000 7f36d6f47640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36d0153ea0 0x7f36d00ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.970+0000 7f36d6f47640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36d0154280 0x7f36d015d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.970+0000 7f36d6f47640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d00ce650 con 0x7f36d0070850 2023-11-24T06:17:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.970+0000 7f36d6f47640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d015ddd0 con 0x7f36d0154280 2023-11-24T06:17:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.970+0000 7f36d6f47640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d015e0a0 con 0x7f36d0153ea0 2023-11-24T06:17:18.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.971+0000 7f36d5744640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36d0154280 0x7f36d015d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.971+0000 7f36d5f45640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36d0153ea0 0x7f36d00ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.971+0000 7f36d5744640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f36d0154280 0x7f36d015d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50364/0 (socket says 172.21.15.5:50364) 2023-11-24T06:17:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.971+0000 7f36d5744640 1 -- 172.21.15.5:0/3146139413 learned_addr learned my addr 172.21.15.5:0/3146139413 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.971+0000 7f36d5744640 1 -- 172.21.15.5:0/3146139413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36d0153ea0 msgr2=0x7f36d00ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.971+0000 7f36d5744640 1 --2- 172.21.15.5:0/3146139413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36d0153ea0 0x7f36d00ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.971+0000 7f36d5744640 1 -- 172.21.15.5:0/3146139413 >> v1:172.21.15.6:6789/0 conn(0x7f36d0070850 legacy=0x7f36d0071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.972+0000 7f36d5744640 1 -- 172.21.15.5:0/3146139413 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d015e380 con 0x7f36d0154280 2023-11-24T06:17:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.973+0000 7f36d5744640 1 --2- 172.21.15.5:0/3146139413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d0154280 0x7f36d015d890 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009870 tx=0x7f36c402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d96717929a1e76 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:18.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.973+0000 7f36d4f43640 1 -- 172.21.15.5:0/3146139413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c4030840 con 0x7f36d0154280 2023-11-24T06:17:18.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.973+0000 7f36d4f43640 1 -- 172.21.15.5:0/3146139413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c40309a0 con 0x7f36d0154280 2023-11-24T06:17:18.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.973+0000 7f36d4f43640 1 -- 172.21.15.5:0/3146139413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c4030d30 con 0x7f36d0154280 2023-11-24T06:17:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.974+0000 7f36d6f47640 1 -- 172.21.15.5:0/3146139413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d0154280 msgr2=0x7f36d015d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.974+0000 7f36d6f47640 1 --2- 172.21.15.5:0/3146139413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d0154280 0x7f36d015d890 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009870 tx=0x7f36c402b980 comp rx=0 tx=0).stop 2023-11-24T06:17:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.974+0000 7f36d6f47640 1 -- 172.21.15.5:0/3146139413 shutdown_connections 2023-11-24T06:17:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.974+0000 7f36d6f47640 1 --2- 172.21.15.5:0/3146139413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d0154280 0x7f36d015d890 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.974+0000 7f36d6f47640 1 --2- 172.21.15.5:0/3146139413 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36d0153ea0 0x7f36d00ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.974+0000 7f36d6f47640 1 -- 172.21.15.5:0/3146139413 >> 172.21.15.5:0/3146139413 conn(0x7f36d00cec10 msgr2=0x7f36d00cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.974+0000 7f36d6f47640 1 -- 172.21.15.5:0/3146139413 shutdown_connections 2023-11-24T06:17:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.974+0000 7f36d6f47640 1 -- 172.21.15.5:0/3146139413 wait complete. 2023-11-24T06:17:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.976+0000 7f36d6f47640 1 Processor -- start 2023-11-24T06:17:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.976+0000 7f36d6f47640 1 -- start start 2023-11-24T06:17:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.978+0000 7f36d6f47640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36d0070850 0x7f36d014dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.978+0000 7f36d6f47640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d0153ea0 0x7f36d01425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.978+0000 7f36d6f47640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36d0154280 0x7f36d00b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.978+0000 7f36d6f47640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d01f6ff0 con 0x7f36d0153ea0 2023-11-24T06:17:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.978+0000 7f36d6f47640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d01f6d10 con 0x7f36d0070850 2023-11-24T06:17:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.978+0000 7f36d6f47640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d00aeb60 con 0x7f36d0154280 2023-11-24T06:17:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.978+0000 7f36d5744640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d0153ea0 0x7f36d01425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.978+0000 7f36d5744640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d0153ea0 0x7f36d01425e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50374/0 (socket says 172.21.15.5:50374) 2023-11-24T06:17:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.978+0000 7f36d5744640 1 -- 172.21.15.5:0/2688398700 learned_addr learned my addr 172.21.15.5:0/2688398700 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:18.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.978+0000 7f36d5f45640 1 --2- 172.21.15.5:0/2688398700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36d0070850 0x7f36d014dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.979+0000 7f36d6746640 1 --2- 172.21.15.5:0/2688398700 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36d0154280 0x7f36d00b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.979+0000 7f36d5744640 1 -- 172.21.15.5:0/2688398700 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36d0154280 msgr2=0x7f36d00b0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.979+0000 7f36d5744640 1 --2- 172.21.15.5:0/2688398700 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36d0154280 0x7f36d00b0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.979+0000 7f36d5744640 1 -- 172.21.15.5:0/2688398700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36d0070850 msgr2=0x7f36d014dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.979+0000 7f36d5744640 1 --2- 172.21.15.5:0/2688398700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36d0070850 0x7f36d014dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.979+0000 7f36d5744640 1 -- 172.21.15.5:0/2688398700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c40094d0 con 0x7f36d0153ea0 2023-11-24T06:17:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.979+0000 7f36d5744640 1 --2- 172.21.15.5:0/2688398700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d0153ea0 0x7f36d01425e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f36c4000c00 tx=0x7f36c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.979+0000 7f36c27fc640 1 -- 172.21.15.5:0/2688398700 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c4039a90 con 0x7f36d0153ea0 2023-11-24T06:17:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.979+0000 7f36c27fc640 1 -- 172.21.15.5:0/2688398700 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c4039bf0 con 0x7f36d0153ea0 2023-11-24T06:17:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.979+0000 7f36c27fc640 1 -- 172.21.15.5:0/2688398700 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c4043410 con 0x7f36d0153ea0 2023-11-24T06:17:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.979+0000 7f36d6f47640 1 -- 172.21.15.5:0/2688398700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d00aee40 con 0x7f36d0153ea0 2023-11-24T06:17:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.980+0000 7f36d6f47640 1 -- 172.21.15.5:0/2688398700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d014e000 con 0x7f36d0153ea0 2023-11-24T06:17:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.981+0000 7f36d6f47640 1 -- 172.21.15.5:0/2688398700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d00ae9c0 con 0x7f36d0153ea0 2023-11-24T06:17:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.982+0000 7f36c27fc640 1 -- 172.21.15.5:0/2688398700 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f36c404c3b0 con 0x7f36d0153ea0 2023-11-24T06:17:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.983+0000 7f36c27fc640 1 --2- 172.21.15.5:0/2688398700 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f36b406ae90 0x7f36b406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.983+0000 7f36c27fc640 1 -- 172.21.15.5:0/2688398700 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36c40abb90 con 0x7f36d0153ea0 2023-11-24T06:17:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.984+0000 7f36d5f45640 1 --2- 172.21.15.5:0/2688398700 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f36b406ae90 0x7f36b406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.984+0000 7f36d5f45640 1 --2- 172.21.15.5:0/2688398700 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f36b406ae90 0x7f36b406d350 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009ad0 tx=0x7f36cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:18.988+0000 7f36c27fc640 1 -- 172.21.15.5:0/2688398700 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36d00ae9c0 con 0x7f36d0153ea0 2023-11-24T06:17:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:19.227+0000 7f36d6f47640 1 -- 172.21.15.5:0/2688398700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f36d00af360 con 0x7f36d0153ea0 2023-11-24T06:17:19.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:19.826+0000 7f36c27fc640 1 -- 172.21.15.5:0/2688398700 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f36d00af360 con 0x7f36d0153ea0 2023-11-24T06:17:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:19.830+0000 7f36d6f47640 1 -- 172.21.15.5:0/2688398700 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f36b406ae90 msgr2=0x7f36b406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:19.830+0000 7f36d6f47640 1 --2- 172.21.15.5:0/2688398700 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f36b406ae90 0x7f36b406d350 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f36cc009ad0 tx=0x7f36cc008040 comp rx=0 tx=0).stop 2023-11-24T06:17:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:19.830+0000 7f36d6f47640 1 -- 172.21.15.5:0/2688398700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d0153ea0 msgr2=0x7f36d01425e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:19.830+0000 7f36d6f47640 1 --2- 172.21.15.5:0/2688398700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d0153ea0 0x7f36d01425e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f36c4000c00 tx=0x7f36c4004730 comp rx=0 tx=0).stop 2023-11-24T06:17:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:19.830+0000 7f36d6f47640 1 -- 172.21.15.5:0/2688398700 shutdown_connections 2023-11-24T06:17:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:19.830+0000 7f36d6f47640 1 --2- 172.21.15.5:0/2688398700 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f36b406ae90 0x7f36b406d350 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:19.830+0000 7f36d6f47640 1 --2- 172.21.15.5:0/2688398700 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f36d0154280 0x7f36d00b0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:19.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:19.830+0000 7f36d6f47640 1 --2- 172.21.15.5:0/2688398700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36d0153ea0 0x7f36d01425e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:19.830+0000 7f36d6f47640 1 --2- 172.21.15.5:0/2688398700 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f36d0070850 0x7f36d014dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:19.830+0000 7f36d6f47640 1 -- 172.21.15.5:0/2688398700 >> 172.21.15.5:0/2688398700 conn(0x7f36d00cec10 msgr2=0x7f36d00ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:19.830+0000 7f36d6f47640 1 -- 172.21.15.5:0/2688398700 shutdown_connections 2023-11-24T06:17:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:19.830+0000 7f36d6f47640 1 -- 172.21.15.5:0/2688398700 wait complete. 2023-11-24T06:17:19.853 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T06:17:19.853 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T06:17:20.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.068+0000 7fef80ad4640 1 Processor -- start 2023-11-24T06:17:20.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.069+0000 7fef80ad4640 1 -- start start 2023-11-24T06:17:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.070+0000 7fef80ad4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef7c15b480 0x7fef7c071890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.070+0000 7fef80ad4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fef7c06bb60 0x7fef7c0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.070+0000 7fef80ad4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7c0cce00 con 0x7fef7c15b0a0 2023-11-24T06:17:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.070+0000 7fef80ad4640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef7c071dd0 con 0x7fef7c06bb60 2023-11-24T06:17:20.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.070+0000 7fef80ad4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0ce560 con 0x7fef7c15b480 2023-11-24T06:17:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.071+0000 7fef7ad9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fef7c06bb60 0x7fef7c0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.071+0000 7fef7ad9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fef7c06bb60 0x7fef7c0b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50378/0 (socket says 172.21.15.5:50378) 2023-11-24T06:17:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.071+0000 7fef7ad9d640 1 -- 172.21.15.5:0/3490469406 learned_addr learned my addr 172.21.15.5:0/3490469406 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.071+0000 7fef73fff640 1 --2- 172.21.15.5:0/3490469406 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef7c15b480 0x7fef7c071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.071+0000 7fef7ad9d640 1 -- 172.21.15.5:0/3490469406 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef7c15b480 msgr2=0x7fef7c071890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.071+0000 7fef7ad9d640 1 --2- 172.21.15.5:0/3490469406 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef7c15b480 0x7fef7c071890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.071+0000 7fef7ad9d640 1 -- 172.21.15.5:0/3490469406 >> v1:172.21.15.6:6789/0 conn(0x7fef7c15b0a0 legacy=0x7fef7c06b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.071+0000 7fef7ad9d640 1 -- 172.21.15.5:0/3490469406 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c0ce840 con 0x7fef7c06bb60 2023-11-24T06:17:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.073+0000 7fef7ad9d640 1 --2- 172.21.15.5:0/3490469406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c06bb60 0x7fef7c0b2720 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fef74009e50 tx=0x7fef7402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea42383f99f834c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:20.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.073+0000 7fef79d9b640 1 -- 172.21.15.5:0/3490469406 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7402ca80 con 0x7fef7c06bb60 2023-11-24T06:17:20.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.073+0000 7fef79d9b640 1 -- 172.21.15.5:0/3490469406 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7402cbe0 con 0x7fef7c06bb60 2023-11-24T06:17:20.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.073+0000 7fef79d9b640 1 -- 172.21.15.5:0/3490469406 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef740343f0 con 0x7fef7c06bb60 2023-11-24T06:17:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.073+0000 7fef80ad4640 1 -- 172.21.15.5:0/3490469406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c06bb60 msgr2=0x7fef7c0b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.073+0000 7fef80ad4640 1 --2- 172.21.15.5:0/3490469406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c06bb60 0x7fef7c0b2720 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fef74009e50 tx=0x7fef7402bee0 comp rx=0 tx=0).stop 2023-11-24T06:17:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.074+0000 7fef80ad4640 1 -- 172.21.15.5:0/3490469406 shutdown_connections 2023-11-24T06:17:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.074+0000 7fef80ad4640 1 --2- 172.21.15.5:0/3490469406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c06bb60 0x7fef7c0b2720 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.074+0000 7fef80ad4640 1 --2- 172.21.15.5:0/3490469406 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef7c15b480 0x7fef7c071890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.074+0000 7fef80ad4640 1 -- 172.21.15.5:0/3490469406 >> 172.21.15.5:0/3490469406 conn(0x7fef7c0cfd60 msgr2=0x7fef7c070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.074+0000 7fef80ad4640 1 -- 172.21.15.5:0/3490469406 shutdown_connections 2023-11-24T06:17:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.074+0000 7fef80ad4640 1 -- 172.21.15.5:0/3490469406 wait complete. 2023-11-24T06:17:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.076+0000 7fef80ad4640 1 Processor -- start 2023-11-24T06:17:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.076+0000 7fef80ad4640 1 -- start start 2023-11-24T06:17:20.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef80ad4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef7c06bb60 0x7fef7c14d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef80ad4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef7c15b0a0 0x7fef7c0b3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef80ad4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c0b3a50 0x7fef7c0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef80ad4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0b3f00 con 0x7fef7c0b3a50 2023-11-24T06:17:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef80ad4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0ad8f0 con 0x7fef7c06bb60 2023-11-24T06:17:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef80ad4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0ad460 con 0x7fef7c15b0a0 2023-11-24T06:17:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef7ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c0b3a50 0x7fef7c0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef7ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c0b3a50 0x7fef7c0ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50394/0 (socket says 172.21.15.5:50394) 2023-11-24T06:17:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef7ad9d640 1 -- 172.21.15.5:0/105527267 learned_addr learned my addr 172.21.15.5:0/105527267 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:20.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef7a59c640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef7c06bb60 0x7fef7c14d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef73fff640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef7c15b0a0 0x7fef7c0b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef7ad9d640 1 -- 172.21.15.5:0/105527267 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef7c15b0a0 msgr2=0x7fef7c0b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef7ad9d640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef7c15b0a0 0x7fef7c0b3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef7ad9d640 1 -- 172.21.15.5:0/105527267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef7c06bb60 msgr2=0x7fef7c14d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef7ad9d640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef7c06bb60 0x7fef7c14d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef7ad9d640 1 -- 172.21.15.5:0/105527267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef74009b30 con 0x7fef7c0b3a50 2023-11-24T06:17:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.077+0000 7fef7a59c640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef7c06bb60 0x7fef7c14d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.078+0000 7fef7ad9d640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c0b3a50 0x7fef7c0ade00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fef74009e50 tx=0x7fef74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.078+0000 7fef72ffd640 1 -- 172.21.15.5:0/105527267 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7403e450 con 0x7fef7c0b3a50 2023-11-24T06:17:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.078+0000 7fef72ffd640 1 -- 172.21.15.5:0/105527267 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7403e5b0 con 0x7fef7c0b3a50 2023-11-24T06:17:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.078+0000 7fef72ffd640 1 -- 172.21.15.5:0/105527267 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7403e960 con 0x7fef7c0b3a50 2023-11-24T06:17:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.078+0000 7fef80ad4640 1 -- 172.21.15.5:0/105527267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c0ac870 con 0x7fef7c0b3a50 2023-11-24T06:17:20.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.078+0000 7fef80ad4640 1 -- 172.21.15.5:0/105527267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c0ace10 con 0x7fef7c0b3a50 2023-11-24T06:17:20.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.080+0000 7fef80ad4640 1 -- 172.21.15.5:0/105527267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7c0ad5d0 con 0x7fef7c0b3a50 2023-11-24T06:17:20.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.081+0000 7fef72ffd640 1 -- 172.21.15.5:0/105527267 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef7402cd50 con 0x7fef7c0b3a50 2023-11-24T06:17:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.082+0000 7fef72ffd640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fef5c06ae70 0x7fef5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.082+0000 7fef72ffd640 1 -- 172.21.15.5:0/105527267 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef740ac0d0 con 0x7fef7c0b3a50 2023-11-24T06:17:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.082+0000 7fef7a59c640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fef5c06ae70 0x7fef5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.083+0000 7fef7a59c640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fef5c06ae70 0x7fef5c06d330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fef6c001680 tx=0x7fef6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:20.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.087+0000 7fef72ffd640 1 -- 172.21.15.5:0/105527267 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef74077180 con 0x7fef7c0b3a50 2023-11-24T06:17:20.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.324+0000 7fef80ad4640 1 -- 172.21.15.5:0/105527267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fef7c072620 con 0x7fef7c0b3a50 2023-11-24T06:17:20.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.829+0000 7fef72ffd640 1 -- 172.21.15.5:0/105527267 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7fef7407a7a0 con 0x7fef7c0b3a50 2023-11-24T06:17:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.834+0000 7fef80ad4640 1 -- 172.21.15.5:0/105527267 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fef5c06ae70 msgr2=0x7fef5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.834+0000 7fef80ad4640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fef5c06ae70 0x7fef5c06d330 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fef6c001680 tx=0x7fef6c009040 comp rx=0 tx=0).stop 2023-11-24T06:17:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.834+0000 7fef80ad4640 1 -- 172.21.15.5:0/105527267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c0b3a50 msgr2=0x7fef7c0ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.834+0000 7fef80ad4640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c0b3a50 0x7fef7c0ade00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fef74009e50 tx=0x7fef74004a40 comp rx=0 tx=0).stop 2023-11-24T06:17:20.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.834+0000 7fef80ad4640 1 -- 172.21.15.5:0/105527267 shutdown_connections 2023-11-24T06:17:20.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.834+0000 7fef80ad4640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fef5c06ae70 0x7fef5c06d330 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:20.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.834+0000 7fef80ad4640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fef7c15b0a0 0x7fef7c0b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:20.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.834+0000 7fef80ad4640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef7c0b3a50 0x7fef7c0ade00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:20.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.834+0000 7fef80ad4640 1 --2- 172.21.15.5:0/105527267 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fef7c06bb60 0x7fef7c14d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:20.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.834+0000 7fef80ad4640 1 -- 172.21.15.5:0/105527267 >> 172.21.15.5:0/105527267 conn(0x7fef7c0cfd60 msgr2=0x7fef7c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:20.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.834+0000 7fef80ad4640 1 -- 172.21.15.5:0/105527267 shutdown_connections 2023-11-24T06:17:20.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:20.834+0000 7fef80ad4640 1 -- 172.21.15.5:0/105527267 wait complete. 2023-11-24T06:17:20.860 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T06:17:20.860 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:17:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.076+0000 7fb01691d640 1 Processor -- start 2023-11-24T06:17:21.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.076+0000 7fb01691d640 1 -- start start 2023-11-24T06:17:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.077+0000 7fb01691d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb010070780 0x7fb01006d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.077+0000 7fb01691d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb010071cd0 con 0x7fb010160970 2023-11-24T06:17:21.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.077+0000 7fb01691d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0100b2120 con 0x7fb010070cd0 2023-11-24T06:17:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.077+0000 7fb01691d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0100b2400 con 0x7fb010070780 2023-11-24T06:17:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.078+0000 7fb00ffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb010160970 0x7fb0100b2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50856/0 (socket says 172.21.15.5:50856) 2023-11-24T06:17:21.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.078+0000 7fb00ffff640 1 -- 172.21.15.5:0/897108753 learned_addr learned my addr 172.21.15.5:0/897108753 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.078+0000 7fb00f7fe640 1 --2- 172.21.15.5:0/897108753 >> v2:172.21.15.6:3300/0 conn(0x7fb010070780 0x7fb01006d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:21.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.078+0000 7fb00effd640 1 -- 172.21.15.5:0/897108753 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959580589 0 0) 0x7fb010071cd0 con 0x7fb010160970 2023-11-24T06:17:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.078+0000 7fb00effd640 1 -- 172.21.15.5:0/897108753 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faff8003520 con 0x7fb010160970 2023-11-24T06:17:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00effd640 1 -- 172.21.15.5:0/897108753 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3578375853 0 0) 0x7faff8003520 con 0x7fb010160970 2023-11-24T06:17:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00effd640 1 -- 172.21.15.5:0/897108753 >> v2:172.21.15.6:3300/0 conn(0x7fb010070780 msgr2=0x7fb01006d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00effd640 1 --2- 172.21.15.5:0/897108753 >> v2:172.21.15.6:3300/0 conn(0x7fb010070780 0x7fb01006d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00effd640 1 -- 172.21.15.5:0/897108753 >> v1:172.21.15.6:6789/0 conn(0x7fb010070cd0 legacy=0x7fb0100acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00effd640 1 -- 172.21.15.5:0/897108753 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0100b26e0 con 0x7fb010160970 2023-11-24T06:17:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00effd640 1 -- 172.21.15.5:0/897108753 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fb0000019f0 con 0x7fb010160970 2023-11-24T06:17:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00f7fe640 1 --2- 172.21.15.5:0/897108753 >> v2:172.21.15.6:3300/0 conn(0x7fb010070780 0x7fb01006d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:17:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00effd640 1 -- 172.21.15.5:0/897108753 >> v1:172.21.15.5:6789/0 conn(0x7fb010160970 legacy=0x7fb0100b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00effd640 1 --2- 172.21.15.5:0/897108753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faff8003970 0x7faff8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00effd640 1 --2- 172.21.15.5:0/897108753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faff80047c0 0x7faff8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00effd640 1 --2- 172.21.15.5:0/897108753 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faff80052a0 0x7faff800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00effd640 1 -- 172.21.15.5:0/897108753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0100b2400 con 0x7faff80047c0 2023-11-24T06:17:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00effd640 1 -- 172.21.15.5:0/897108753 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff800c170 con 0x7faff8003970 2023-11-24T06:17:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00effd640 1 -- 172.21.15.5:0/897108753 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff800c330 con 0x7faff80052a0 2023-11-24T06:17:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.079+0000 7fb00f7fe640 1 --2- 172.21.15.5:0/897108753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faff80047c0 0x7faff8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.080+0000 7fb00ffff640 1 --2- 172.21.15.5:0/897108753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faff8003970 0x7faff8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.080+0000 7fb00f7fe640 1 -- 172.21.15.5:0/897108753 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faff80052a0 msgr2=0x7faff800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.080+0000 7fb00f7fe640 1 --2- 172.21.15.5:0/897108753 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7faff80052a0 0x7faff800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.080+0000 7fb00f7fe640 1 -- 172.21.15.5:0/897108753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faff8003970 msgr2=0x7faff8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.080+0000 7fb00f7fe640 1 --2- 172.21.15.5:0/897108753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faff8003970 0x7faff8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.080+0000 7fb00f7fe640 1 -- 172.21.15.5:0/897108753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff8003520 con 0x7faff80047c0 2023-11-24T06:17:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.080+0000 7fb00ffff640 1 --2- 172.21.15.5:0/897108753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faff8003970 0x7faff8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.080+0000 7fb014eba640 1 -- 172.21.15.5:0/897108753 reap_dead start 2023-11-24T06:17:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.081+0000 7fb00f7fe640 1 --2- 172.21.15.5:0/897108753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faff80047c0 0x7faff8004c40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb004006fd0 tx=0x7faff80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b82f51a77612a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.081+0000 7fb00effd640 1 -- 172.21.15.5:0/897108753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004004230 con 0x7faff80047c0 2023-11-24T06:17:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.081+0000 7fb00effd640 1 -- 172.21.15.5:0/897108753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb004004390 con 0x7faff80047c0 2023-11-24T06:17:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.081+0000 7fb00effd640 1 -- 172.21.15.5:0/897108753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004003830 con 0x7faff80047c0 2023-11-24T06:17:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.082+0000 7fb01691d640 1 -- 172.21.15.5:0/897108753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faff80047c0 msgr2=0x7faff8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.082+0000 7fb01691d640 1 --2- 172.21.15.5:0/897108753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faff80047c0 0x7faff8004c40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb004006fd0 tx=0x7faff80057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.082+0000 7fb01691d640 1 -- 172.21.15.5:0/897108753 shutdown_connections 2023-11-24T06:17:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.082+0000 7fb01691d640 1 --2- 172.21.15.5:0/897108753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faff80047c0 0x7faff8004c40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.082+0000 7fb01691d640 1 -- 172.21.15.5:0/897108753 >> 172.21.15.5:0/897108753 conn(0x7fb0100b0d80 msgr2=0x7fb0100b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.082+0000 7fb01691d640 1 -- 172.21.15.5:0/897108753 shutdown_connections 2023-11-24T06:17:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.082+0000 7fb01691d640 1 -- 172.21.15.5:0/897108753 wait complete. 2023-11-24T06:17:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.084+0000 7fb01691d640 1 Processor -- start 2023-11-24T06:17:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.084+0000 7fb01691d640 1 -- start start 2023-11-24T06:17:21.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.085+0000 7fb01691d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faff80047c0 0x7fb010141800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:21.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.085+0000 7fb01691d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb010145010 0x7fb010156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:21.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.085+0000 7fb01691d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb01015db50 0x7fb01009c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:21.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.085+0000 7fb01691d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01015d6c0 con 0x7fb010145010 2023-11-24T06:17:21.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.085+0000 7fb01691d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb010141d40 con 0x7faff80047c0 2023-11-24T06:17:21.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.085+0000 7fb01691d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01015cab0 con 0x7fb01015db50 2023-11-24T06:17:21.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.086+0000 7fb00f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb010145010 0x7fb010156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:21.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.086+0000 7fb00f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb010145010 0x7fb010156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50408/0 (socket says 172.21.15.5:50408) 2023-11-24T06:17:21.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.086+0000 7fb00f7fe640 1 -- 172.21.15.5:0/1811056367 learned_addr learned my addr 172.21.15.5:0/1811056367 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:21.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.086+0000 7fb00ffff640 1 --2- 172.21.15.5:0/1811056367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faff80047c0 0x7fb010141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:21.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.086+0000 7fb014eba640 1 --2- 172.21.15.5:0/1811056367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb01015db50 0x7fb01009c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:21.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.086+0000 7fb00f7fe640 1 -- 172.21.15.5:0/1811056367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb01015db50 msgr2=0x7fb01009c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:21.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.086+0000 7fb00f7fe640 1 --2- 172.21.15.5:0/1811056367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb01015db50 0x7fb01009c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.086+0000 7fb00f7fe640 1 -- 172.21.15.5:0/1811056367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faff80047c0 msgr2=0x7fb010141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:21.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.086+0000 7fb00f7fe640 1 --2- 172.21.15.5:0/1811056367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faff80047c0 0x7fb010141800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.086+0000 7fb00f7fe640 1 -- 172.21.15.5:0/1811056367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01015cd90 con 0x7fb010145010 2023-11-24T06:17:21.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.087+0000 7fb00f7fe640 1 --2- 172.21.15.5:0/1811056367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb010145010 0x7fb010156730 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb00402be80 tx=0x7fb004004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.087+0000 7fb00cff9640 1 -- 172.21.15.5:0/1811056367 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004043600 con 0x7fb010145010 2023-11-24T06:17:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.087+0000 7fb00cff9640 1 -- 172.21.15.5:0/1811056367 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb004043760 con 0x7fb010145010 2023-11-24T06:17:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.087+0000 7fb01691d640 1 -- 172.21.15.5:0/1811056367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01015d070 con 0x7fb010145010 2023-11-24T06:17:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.087+0000 7fb00cff9640 1 -- 172.21.15.5:0/1811056367 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004043b10 con 0x7fb010145010 2023-11-24T06:17:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.087+0000 7fb01691d640 1 -- 172.21.15.5:0/1811056367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb010156c70 con 0x7fb010145010 2023-11-24T06:17:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.088+0000 7fb01691d640 1 -- 172.21.15.5:0/1811056367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb01015d870 con 0x7fb010145010 2023-11-24T06:17:21.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.089+0000 7fb00cff9640 1 -- 172.21.15.5:0/1811056367 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb0040383f0 con 0x7fb010145010 2023-11-24T06:17:21.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.091+0000 7fb00cff9640 1 --2- 172.21.15.5:0/1811056367 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7faffc06ae70 0x7faffc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:21.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.091+0000 7fb00cff9640 1 -- 172.21.15.5:0/1811056367 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0040ab6d0 con 0x7fb010145010 2023-11-24T06:17:21.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.091+0000 7fb00ffff640 1 --2- 172.21.15.5:0/1811056367 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7faffc06ae70 0x7faffc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:21.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.092+0000 7fb00ffff640 1 --2- 172.21.15.5:0/1811056367 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7faffc06ae70 0x7faffc06d330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb000001b20 tx=0x7fb000009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:21.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.096+0000 7fb00cff9640 1 -- 172.21.15.5:0/1811056367 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb01015d870 con 0x7fb010145010 2023-11-24T06:17:21.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.321+0000 7fb01691d640 1 -- 172.21.15.5:0/1811056367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb010074770 con 0x7fb010145010 2023-11-24T06:17:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.323+0000 7fb00cff9640 1 -- 172.21.15.5:0/1811056367 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4901 (secure 0 0 0) 0x7fb00407a170 con 0x7fb010145010 2023-11-24T06:17:21.326 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:21.326 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6833/2072679798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2072679798},{"type":"v1","addr":"172.21.15.6:6833","nonce":2072679798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3855353235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3855353235},{"type":"v1","addr":"172.21.15.6:6835","nonce":3855353235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2394562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2394562290},{"type":"v1","addr":"172.21.15.5:6836","nonce":2394562290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:17:16.874899+0000","modified":"2023-11-24T06:17:20.326601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5217":{"gid":5217,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/3690265514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3690265514},{"type":"v1","addr":"172.21.15.5:6837","nonce":3690265514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:17:21.326 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2023-11-24T06:17:21.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.325+0000 7fb01691d640 1 -- 172.21.15.5:0/1811056367 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7faffc06ae70 msgr2=0x7faffc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:21.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.325+0000 7fb01691d640 1 --2- 172.21.15.5:0/1811056367 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7faffc06ae70 0x7faffc06d330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb000001b20 tx=0x7fb000009040 comp rx=0 tx=0).stop 2023-11-24T06:17:21.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.325+0000 7fb01691d640 1 -- 172.21.15.5:0/1811056367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb010145010 msgr2=0x7fb010156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:21.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.325+0000 7fb01691d640 1 --2- 172.21.15.5:0/1811056367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb010145010 0x7fb010156730 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb00402be80 tx=0x7fb004004730 comp rx=0 tx=0).stop 2023-11-24T06:17:21.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.325+0000 7fb01691d640 1 -- 172.21.15.5:0/1811056367 shutdown_connections 2023-11-24T06:17:21.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.325+0000 7fb01691d640 1 --2- 172.21.15.5:0/1811056367 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7faffc06ae70 0x7faffc06d330 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.325+0000 7fb01691d640 1 --2- 172.21.15.5:0/1811056367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb01015db50 0x7fb01009c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.325+0000 7fb01691d640 1 --2- 172.21.15.5:0/1811056367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb010145010 0x7fb010156730 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.325+0000 7fb01691d640 1 --2- 172.21.15.5:0/1811056367 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7faff80047c0 0x7fb010141800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.325+0000 7fb01691d640 1 -- 172.21.15.5:0/1811056367 >> 172.21.15.5:0/1811056367 conn(0x7fb0100b0d80 msgr2=0x7fb0100af020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:21.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.325+0000 7fb01691d640 1 -- 172.21.15.5:0/1811056367 shutdown_connections 2023-11-24T06:17:21.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.325+0000 7fb01691d640 1 -- 172.21.15.5:0/1811056367 wait complete. 2023-11-24T06:17:21.346 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:17:21.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.560+0000 7fc5129f4640 1 Processor -- start 2023-11-24T06:17:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.560+0000 7fc5129f4640 1 -- start start 2023-11-24T06:17:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.562+0000 7fc5129f4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc50c13dfd0 0x7fc50c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.562+0000 7fc5129f4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc50c0cf4d0 0x7fc50c06c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.562+0000 7fc5129f4640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50c0ce7c0 con 0x7fc50c0cf050 2023-11-24T06:17:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.562+0000 7fc5129f4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50c140900 con 0x7fc50c0cf4d0 2023-11-24T06:17:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.562+0000 7fc5129f4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c15fa70 con 0x7fc50c13dfd0 2023-11-24T06:17:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.563+0000 7fc5119f2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc50c13dfd0 0x7fc50c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.563+0000 7fc5121f3640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc50c0cf4d0 0x7fc50c06c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.563+0000 7fc5121f3640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc50c0cf4d0 0x7fc50c06c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:33824/0 (socket says 172.21.15.5:33824) 2023-11-24T06:17:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.563+0000 7fc5121f3640 1 -- 172.21.15.5:0/3709717814 learned_addr learned my addr 172.21.15.5:0/3709717814 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.563+0000 7fc5119f2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc50c13dfd0 0x7fc50c15be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:54630/0 (socket says 172.21.15.5:54630) 2023-11-24T06:17:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.564+0000 7fc5121f3640 1 -- 172.21.15.5:0/3709717814 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc50c13dfd0 msgr2=0x7fc50c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.564+0000 7fc5121f3640 1 --2- 172.21.15.5:0/3709717814 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc50c13dfd0 0x7fc50c15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.564+0000 7fc5121f3640 1 -- 172.21.15.5:0/3709717814 >> v1:172.21.15.6:6789/0 conn(0x7fc50c0cf050 legacy=0x7fc50c06c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.564+0000 7fc5121f3640 1 -- 172.21.15.5:0/3709717814 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c15ee60 con 0x7fc50c0cf4d0 2023-11-24T06:17:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.564+0000 7fc5119f2640 1 --2- 172.21.15.5:0/3709717814 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc50c13dfd0 0x7fc50c15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:17:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.565+0000 7fc5121f3640 1 --2- 172.21.15.5:0/3709717814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc50c0cf4d0 0x7fc50c06c840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc504009e50 tx=0x7fc50402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c7fa1b6ecd20f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.565+0000 7fc5109f0640 1 -- 172.21.15.5:0/3709717814 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50402cb00 con 0x7fc50c0cf4d0 2023-11-24T06:17:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.566+0000 7fc5109f0640 1 -- 172.21.15.5:0/3709717814 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50402cc60 con 0x7fc50c0cf4d0 2023-11-24T06:17:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.566+0000 7fc5109f0640 1 -- 172.21.15.5:0/3709717814 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5040343f0 con 0x7fc50c0cf4d0 2023-11-24T06:17:21.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.566+0000 7fc5129f4640 1 -- 172.21.15.5:0/3709717814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc50c0cf4d0 msgr2=0x7fc50c06c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:21.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.566+0000 7fc5129f4640 1 --2- 172.21.15.5:0/3709717814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc50c0cf4d0 0x7fc50c06c840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc504009e50 tx=0x7fc50402bf60 comp rx=0 tx=0).stop 2023-11-24T06:17:21.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.566+0000 7fc5129f4640 1 -- 172.21.15.5:0/3709717814 shutdown_connections 2023-11-24T06:17:21.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.566+0000 7fc5129f4640 1 --2- 172.21.15.5:0/3709717814 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc50c0cf4d0 0x7fc50c06c840 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.566+0000 7fc5129f4640 1 --2- 172.21.15.5:0/3709717814 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc50c13dfd0 0x7fc50c15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.566+0000 7fc5129f4640 1 -- 172.21.15.5:0/3709717814 >> 172.21.15.5:0/3709717814 conn(0x7fc50c06a860 msgr2=0x7fc50c06ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:21.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.566+0000 7fc5129f4640 1 -- 172.21.15.5:0/3709717814 shutdown_connections 2023-11-24T06:17:21.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.566+0000 7fc5129f4640 1 -- 172.21.15.5:0/3709717814 wait complete. 2023-11-24T06:17:21.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.568+0000 7fc5129f4640 1 Processor -- start 2023-11-24T06:17:21.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.568+0000 7fc5129f4640 1 -- start start 2023-11-24T06:17:21.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.569+0000 7fc5129f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc50c0cf050 0x7fc50c131df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:21.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.569+0000 7fc5129f4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc50c0cf4d0 0x7fc50c0714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:21.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.569+0000 7fc5129f4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc50c13dfd0 0x7fc50c12f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:21.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.569+0000 7fc5129f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c132330 con 0x7fc50c0cf050 2023-11-24T06:17:21.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.569+0000 7fc5129f4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c0c9ac0 con 0x7fc50c0cf4d0 2023-11-24T06:17:21.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.569+0000 7fc5129f4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c0c9da0 con 0x7fc50c13dfd0 2023-11-24T06:17:21.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.570+0000 7fc5119f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc50c0cf050 0x7fc50c131df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:21.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.570+0000 7fc5119f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc50c0cf050 0x7fc50c131df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50424/0 (socket says 172.21.15.5:50424) 2023-11-24T06:17:21.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.570+0000 7fc5119f2640 1 -- 172.21.15.5:0/3184829518 learned_addr learned my addr 172.21.15.5:0/3184829518 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:21.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.570+0000 7fc5119f2640 1 -- 172.21.15.5:0/3184829518 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc50c13dfd0 msgr2=0x7fc50c12f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.570+0000 7fc5119f2640 1 --2- 172.21.15.5:0/3184829518 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc50c13dfd0 0x7fc50c12f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.570+0000 7fc5119f2640 1 -- 172.21.15.5:0/3184829518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc50c0cf4d0 msgr2=0x7fc50c0714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.570+0000 7fc5119f2640 1 --2- 172.21.15.5:0/3184829518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc50c0cf4d0 0x7fc50c0714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.570+0000 7fc5119f2640 1 -- 172.21.15.5:0/3184829518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc508009400 con 0x7fc50c0cf050 2023-11-24T06:17:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.571+0000 7fc5119f2640 1 --2- 172.21.15.5:0/3184829518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc50c0cf050 0x7fc50c131df0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc50800ebe0 tx=0x7fc50800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.571+0000 7fc5027fc640 1 -- 172.21.15.5:0/3184829518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc508009760 con 0x7fc50c0cf050 2023-11-24T06:17:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.571+0000 7fc5027fc640 1 -- 172.21.15.5:0/3184829518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50800cc30 con 0x7fc50c0cf050 2023-11-24T06:17:21.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.571+0000 7fc5129f4640 1 -- 172.21.15.5:0/3184829518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc504009b30 con 0x7fc50c0cf050 2023-11-24T06:17:21.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.571+0000 7fc5027fc640 1 -- 172.21.15.5:0/3184829518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50800eee0 con 0x7fc50c0cf050 2023-11-24T06:17:21.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.571+0000 7fc5129f4640 1 -- 172.21.15.5:0/3184829518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50c0ca360 con 0x7fc50c0cf050 2023-11-24T06:17:21.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.572+0000 7fc5129f4640 1 -- 172.21.15.5:0/3184829518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50c06d4e0 con 0x7fc50c0cf050 2023-11-24T06:17:21.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.573+0000 7fc5027fc640 1 -- 172.21.15.5:0/3184829518 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc508013470 con 0x7fc50c0cf050 2023-11-24T06:17:21.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.574+0000 7fc5027fc640 1 --2- 172.21.15.5:0/3184829518 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc4f006ae40 0x7fc4f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:21.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.574+0000 7fc5027fc640 1 -- 172.21.15.5:0/3184829518 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc508089aa0 con 0x7fc50c0cf050 2023-11-24T06:17:21.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.574+0000 7fc5111f1640 1 --2- 172.21.15.5:0/3184829518 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc4f006ae40 0x7fc4f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:21.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.575+0000 7fc5111f1640 1 --2- 172.21.15.5:0/3184829518 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc4f006ae40 0x7fc4f006d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc000f60 tx=0x7fc4fc0079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.579+0000 7fc5027fc640 1 -- 172.21.15.5:0/3184829518 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc508054a30 con 0x7fc50c0cf050 2023-11-24T06:17:21.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.766+0000 7fc5129f4640 1 -- 172.21.15.5:0/3184829518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc50c04e840 con 0x7fc50c0cf050 2023-11-24T06:17:21.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.767+0000 7fc5027fc640 1 -- 172.21.15.5:0/3184829518 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16425 (secure 0 0 0) 0x7fc508058050 con 0x7fc50c0cf050 2023-11-24T06:17:21.771 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:21.771 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:17:18.380558+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:17:14.868917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:17:15.895189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:17:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.770+0000 7fc5129f4640 1 -- 172.21.15.5:0/3184829518 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc4f006ae40 msgr2=0x7fc4f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.770+0000 7fc5129f4640 1 --2- 172.21.15.5:0/3184829518 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc4f006ae40 0x7fc4f006d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc000f60 tx=0x7fc4fc0079b0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.770+0000 7fc5129f4640 1 -- 172.21.15.5:0/3184829518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc50c0cf050 msgr2=0x7fc50c131df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:21.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.770+0000 7fc5129f4640 1 --2- 172.21.15.5:0/3184829518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc50c0cf050 0x7fc50c131df0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc50800ebe0 tx=0x7fc50800c520 comp rx=0 tx=0).stop 2023-11-24T06:17:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.770+0000 7fc5129f4640 1 -- 172.21.15.5:0/3184829518 shutdown_connections 2023-11-24T06:17:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.770+0000 7fc5129f4640 1 --2- 172.21.15.5:0/3184829518 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc4f006ae40 0x7fc4f006d300 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.770+0000 7fc5129f4640 1 --2- 172.21.15.5:0/3184829518 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc50c13dfd0 0x7fc50c12f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.770+0000 7fc5129f4640 1 --2- 172.21.15.5:0/3184829518 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc50c0cf4d0 0x7fc50c0714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.770+0000 7fc5129f4640 1 --2- 172.21.15.5:0/3184829518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc50c0cf050 0x7fc50c131df0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.770+0000 7fc5129f4640 1 -- 172.21.15.5:0/3184829518 >> 172.21.15.5:0/3184829518 conn(0x7fc50c06a860 msgr2=0x7fc50c13eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:21.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.770+0000 7fc5129f4640 1 -- 172.21.15.5:0/3184829518 shutdown_connections 2023-11-24T06:17:21.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:21.770+0000 7fc5129f4640 1 -- 172.21.15.5:0/3184829518 wait complete. 2023-11-24T06:17:21.791 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T06:17:21.791 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:17:22.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.002+0000 7fc8d24df640 1 Processor -- start 2023-11-24T06:17:22.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.003+0000 7fc8d24df640 1 -- start start 2023-11-24T06:17:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.004+0000 7fc8d24df640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc8cc0ce010 0x7fc8cc15d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.004+0000 7fc8d24df640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc8cc06b550 0x7fc8cc06bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.004+0000 7fc8d24df640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8cc0cfbc0 con 0x7fc8cc0cdc30 2023-11-24T06:17:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.004+0000 7fc8d24df640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8cc0b0450 con 0x7fc8cc06b550 2023-11-24T06:17:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.004+0000 7fc8d24df640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8cc0cfe10 con 0x7fc8cc0ce010 2023-11-24T06:17:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.005+0000 7fc8d1cde640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc8cc06b550 0x7fc8cc06bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.005+0000 7fc8d1cde640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc8cc06b550 0x7fc8cc06bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50438/0 (socket says 172.21.15.5:50438) 2023-11-24T06:17:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.005+0000 7fc8d1cde640 1 -- 172.21.15.5:0/1812448311 learned_addr learned my addr 172.21.15.5:0/1812448311 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.005+0000 7fc8d0cdc640 1 --2- 172.21.15.5:0/1812448311 >> v2:172.21.15.6:3300/0 conn(0x7fc8cc0ce010 0x7fc8cc15d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.005+0000 7fc8d1cde640 1 -- 172.21.15.5:0/1812448311 >> v2:172.21.15.6:3300/0 conn(0x7fc8cc0ce010 msgr2=0x7fc8cc15d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.005+0000 7fc8d1cde640 1 --2- 172.21.15.5:0/1812448311 >> v2:172.21.15.6:3300/0 conn(0x7fc8cc0ce010 0x7fc8cc15d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.005+0000 7fc8d1cde640 1 -- 172.21.15.5:0/1812448311 >> v1:172.21.15.5:6789/0 conn(0x7fc8cc0cdc30 legacy=0x7fc8cc15cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.005+0000 7fc8d1cde640 1 -- 172.21.15.5:0/1812448311 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8cc0d00f0 con 0x7fc8cc06b550 2023-11-24T06:17:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.006+0000 7fc8d1cde640 1 --2- 172.21.15.5:0/1812448311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8cc06b550 0x7fc8cc06bc10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0b0360 tx=0x7fc8c402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b475e56e7e7ea3bd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.007+0000 7fc8c3fff640 1 -- 172.21.15.5:0/1812448311 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c402ca80 con 0x7fc8cc06b550 2023-11-24T06:17:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.007+0000 7fc8c3fff640 1 -- 172.21.15.5:0/1812448311 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c402cbe0 con 0x7fc8cc06b550 2023-11-24T06:17:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.007+0000 7fc8c3fff640 1 -- 172.21.15.5:0/1812448311 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40343f0 con 0x7fc8cc06b550 2023-11-24T06:17:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.007+0000 7fc8d24df640 1 -- 172.21.15.5:0/1812448311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8cc06b550 msgr2=0x7fc8cc06bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.007+0000 7fc8d24df640 1 --2- 172.21.15.5:0/1812448311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8cc06b550 0x7fc8cc06bc10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0b0360 tx=0x7fc8c402bee0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.007+0000 7fc8d24df640 1 -- 172.21.15.5:0/1812448311 shutdown_connections 2023-11-24T06:17:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.007+0000 7fc8d24df640 1 --2- 172.21.15.5:0/1812448311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8cc06b550 0x7fc8cc06bc10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.007+0000 7fc8d24df640 1 --2- 172.21.15.5:0/1812448311 >> v2:172.21.15.6:3300/0 conn(0x7fc8cc0ce010 0x7fc8cc15d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.007+0000 7fc8d24df640 1 -- 172.21.15.5:0/1812448311 >> 172.21.15.5:0/1812448311 conn(0x7fc8cc0d0e70 msgr2=0x7fc8cc0d1280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.008+0000 7fc8d24df640 1 -- 172.21.15.5:0/1812448311 shutdown_connections 2023-11-24T06:17:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.008+0000 7fc8d24df640 1 -- 172.21.15.5:0/1812448311 wait complete. 2023-11-24T06:17:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.009+0000 7fc8d24df640 1 Processor -- start 2023-11-24T06:17:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.009+0000 7fc8d24df640 1 -- start start 2023-11-24T06:17:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.010+0000 7fc8d24df640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8cc06b550 0x7fc8cc14c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.010+0000 7fc8d24df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8cc0cdc30 0x7fc8cc160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.010+0000 7fc8d24df640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8cc0ce010 0x7fc8cc1f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.010+0000 7fc8d24df640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc0ae650 con 0x7fc8cc0cdc30 2023-11-24T06:17:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.011+0000 7fc8d24df640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc0caba0 con 0x7fc8cc0ce010 2023-11-24T06:17:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.011+0000 7fc8d24df640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc0ae8a0 con 0x7fc8cc06b550 2023-11-24T06:17:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.011+0000 7fc8d0cdc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8cc0cdc30 0x7fc8cc160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.011+0000 7fc8d0cdc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8cc0cdc30 0x7fc8cc160120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50442/0 (socket says 172.21.15.5:50442) 2023-11-24T06:17:22.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.011+0000 7fc8d0cdc640 1 -- 172.21.15.5:0/593027054 learned_addr learned my addr 172.21.15.5:0/593027054 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.011+0000 7fc8d14dd640 1 --2- 172.21.15.5:0/593027054 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8cc06b550 0x7fc8cc14c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.011+0000 7fc8d1cde640 1 --2- 172.21.15.5:0/593027054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8cc0ce010 0x7fc8cc1f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.011+0000 7fc8d0cdc640 1 -- 172.21.15.5:0/593027054 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8cc06b550 msgr2=0x7fc8cc14c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.011+0000 7fc8d0cdc640 1 --2- 172.21.15.5:0/593027054 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8cc06b550 0x7fc8cc14c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.011+0000 7fc8d0cdc640 1 -- 172.21.15.5:0/593027054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8cc0ce010 msgr2=0x7fc8cc1f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.011+0000 7fc8d0cdc640 1 --2- 172.21.15.5:0/593027054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8cc0ce010 0x7fc8cc1f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.011+0000 7fc8d0cdc640 1 -- 172.21.15.5:0/593027054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c4009b30 con 0x7fc8cc0cdc30 2023-11-24T06:17:22.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.012+0000 7fc8d0cdc640 1 --2- 172.21.15.5:0/593027054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8cc0cdc30 0x7fc8cc160120 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00d7d0 tx=0x7fc8bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:22.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.012+0000 7fc8c1ffb640 1 -- 172.21.15.5:0/593027054 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc00b6c0 con 0x7fc8cc0cdc30 2023-11-24T06:17:22.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.012+0000 7fc8c1ffb640 1 -- 172.21.15.5:0/593027054 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8bc00b820 con 0x7fc8cc0cdc30 2023-11-24T06:17:22.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.012+0000 7fc8c1ffb640 1 -- 172.21.15.5:0/593027054 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc00bbb0 con 0x7fc8cc0cdc30 2023-11-24T06:17:22.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.012+0000 7fc8d24df640 1 -- 172.21.15.5:0/593027054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8cc0aeb80 con 0x7fc8cc0cdc30 2023-11-24T06:17:22.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.012+0000 7fc8d24df640 1 -- 172.21.15.5:0/593027054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8cc160660 con 0x7fc8cc0cdc30 2023-11-24T06:17:22.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.013+0000 7fc8d24df640 1 -- 172.21.15.5:0/593027054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8cc04e840 con 0x7fc8cc0cdc30 2023-11-24T06:17:22.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.014+0000 7fc8c1ffb640 1 -- 172.21.15.5:0/593027054 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc8bc00bd10 con 0x7fc8cc0cdc30 2023-11-24T06:17:22.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.015+0000 7fc8c1ffb640 1 --2- 172.21.15.5:0/593027054 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc8b006ae40 0x7fc8b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:22.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.015+0000 7fc8c1ffb640 1 -- 172.21.15.5:0/593027054 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8bc089c70 con 0x7fc8cc0cdc30 2023-11-24T06:17:22.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.015+0000 7fc8d14dd640 1 --2- 172.21.15.5:0/593027054 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc8b006ae40 0x7fc8b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:22.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.016+0000 7fc8d14dd640 1 --2- 172.21.15.5:0/593027054 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc8b006ae40 0x7fc8b006d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8000f60 tx=0x7fc8c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:22.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.020+0000 7fc8c1ffb640 1 -- 172.21.15.5:0/593027054 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8bc054c80 con 0x7fc8cc0cdc30 2023-11-24T06:17:22.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.208+0000 7fc8d24df640 1 -- 172.21.15.5:0/593027054 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc8cc15d730 con 0x7fc8b006ae40 2023-11-24T06:17:22.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.219+0000 7fc8c1ffb640 1 -- 172.21.15.5:0/593027054 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287046 (secure 0 0 0) 0x7fc8c8001390 con 0x7fc8b006ae40 2023-11-24T06:17:22.226 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:22.227 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-11-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.225+0000 7fc8d24df640 1 -- 172.21.15.5:0/593027054 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc8b006ae40 msgr2=0x7fc8b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.225+0000 7fc8d24df640 1 --2- 172.21.15.5:0/593027054 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc8b006ae40 0x7fc8b006d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8000f60 tx=0x7fc8c8009040 comp rx=0 tx=0).stop 2023-11-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.225+0000 7fc8d24df640 1 -- 172.21.15.5:0/593027054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8cc0cdc30 msgr2=0x7fc8cc160120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.225+0000 7fc8d24df640 1 --2- 172.21.15.5:0/593027054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8cc0cdc30 0x7fc8cc160120 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00d7d0 tx=0x7fc8bc00dc90 comp rx=0 tx=0).stop 2023-11-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.225+0000 7fc8d24df640 1 -- 172.21.15.5:0/593027054 shutdown_connections 2023-11-24T06:17:22.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.225+0000 7fc8d24df640 1 --2- 172.21.15.5:0/593027054 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc8b006ae40 0x7fc8b006d300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.225+0000 7fc8d24df640 1 --2- 172.21.15.5:0/593027054 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc8cc0ce010 0x7fc8cc1f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.225+0000 7fc8d24df640 1 --2- 172.21.15.5:0/593027054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8cc0cdc30 0x7fc8cc160120 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.225+0000 7fc8d24df640 1 --2- 172.21.15.5:0/593027054 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc8cc06b550 0x7fc8cc14c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.225+0000 7fc8d24df640 1 -- 172.21.15.5:0/593027054 >> 172.21.15.5:0/593027054 conn(0x7fc8cc0d0e70 msgr2=0x7fc8cc0b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.225+0000 7fc8d24df640 1 -- 172.21.15.5:0/593027054 shutdown_connections 2023-11-24T06:17:22.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.225+0000 7fc8d24df640 1 -- 172.21.15.5:0/593027054 wait complete. 2023-11-24T06:17:22.247 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-11-24T06:17:22.115806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241192,"kb_used_data":4212,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733528,"statfs":{"total":773094113280,"available":772847132672,"internally_reserved":0,"allocated":4313088,"data_stored":2391141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001825"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820444+0000","last_change":"2023-11-24T06:17:17.425737+0000","last_active":"2023-11-24T06:17:18.820444+0000","last_peered":"2023-11-24T06:17:18.820444+0000","last_clean":"2023-11-24T06:17:18.820444+0000","last_became_active":"2023-11-24T06:17:17.425257+0000","last_became_peered":"2023-11-24T06:17:17.425257+0000","last_unstale":"2023-11-24T06:17:18.820444+0000","last_undegraded":"2023-11-24T06:17:18.820444+0000","last_fullsized":"2023-11-24T06:17:18.820444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:54:03.724108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.388245+0000","last_change":"2023-11-24T06:17:17.415520+0000","last_active":"2023-11-24T06:17:18.388245+0000","last_peered":"2023-11-24T06:17:18.388245+0000","last_clean":"2023-11-24T06:17:18.388245+0000","last_became_active":"2023-11-24T06:17:17.413377+0000","last_became_peered":"2023-11-24T06:17:17.413377+0000","last_unstale":"2023-11-24T06:17:18.388245+0000","last_undegraded":"2023-11-24T06:17:18.388245+0000","last_fullsized":"2023-11-24T06:17:18.388245+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:28.886659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.822352+0000","last_change":"2023-11-24T06:17:17.419104+0000","last_active":"2023-11-24T06:17:18.822352+0000","last_peered":"2023-11-24T06:17:18.822352+0000","last_clean":"2023-11-24T06:17:18.822352+0000","last_became_active":"2023-11-24T06:17:17.418752+0000","last_became_peered":"2023-11-24T06:17:17.418752+0000","last_unstale":"2023-11-24T06:17:18.822352+0000","last_undegraded":"2023-11-24T06:17:18.822352+0000","last_fullsized":"2023-11-24T06:17:18.822352+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:34:30.808706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819988+0000","last_change":"2023-11-24T06:17:17.413918+0000","last_active":"2023-11-24T06:17:18.819988+0000","last_peered":"2023-11-24T06:17:18.819988+0000","last_clean":"2023-11-24T06:17:18.819988+0000","last_became_active":"2023-11-24T06:17:17.413523+0000","last_became_peered":"2023-11-24T06:17:17.413523+0000","last_unstale":"2023-11-24T06:17:18.819988+0000","last_undegraded":"2023-11-24T06:17:18.819988+0000","last_fullsized":"2023-11-24T06:17:18.819988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:12.852430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.816481+0000","last_change":"2023-11-24T06:17:17.423600+0000","last_active":"2023-11-24T06:17:18.816481+0000","last_peered":"2023-11-24T06:17:18.816481+0000","last_clean":"2023-11-24T06:17:18.816481+0000","last_became_active":"2023-11-24T06:17:17.421248+0000","last_became_peered":"2023-11-24T06:17:17.421248+0000","last_unstale":"2023-11-24T06:17:18.816481+0000","last_undegraded":"2023-11-24T06:17:18.816481+0000","last_fullsized":"2023-11-24T06:17:18.816481+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:56:00.416571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387593+0000","last_change":"2023-11-24T06:17:17.417869+0000","last_active":"2023-11-24T06:17:18.387593+0000","last_peered":"2023-11-24T06:17:18.387593+0000","last_clean":"2023-11-24T06:17:18.387593+0000","last_became_active":"2023-11-24T06:17:17.417580+0000","last_became_peered":"2023-11-24T06:17:17.417580+0000","last_unstale":"2023-11-24T06:17:18.387593+0000","last_undegraded":"2023-11-24T06:17:18.387593+0000","last_fullsized":"2023-11-24T06:17:18.387593+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:28.122398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392453+0000","last_change":"2023-11-24T06:17:17.420507+0000","last_active":"2023-11-24T06:17:18.392453+0000","last_peered":"2023-11-24T06:17:18.392453+0000","last_clean":"2023-11-24T06:17:18.392453+0000","last_became_active":"2023-11-24T06:17:17.420168+0000","last_became_peered":"2023-11-24T06:17:17.420168+0000","last_unstale":"2023-11-24T06:17:18.392453+0000","last_undegraded":"2023-11-24T06:17:18.392453+0000","last_fullsized":"2023-11-24T06:17:18.392453+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:36:54.531275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.390465+0000","last_change":"2023-11-24T06:17:17.422042+0000","last_active":"2023-11-24T06:17:18.390465+0000","last_peered":"2023-11-24T06:17:18.390465+0000","last_clean":"2023-11-24T06:17:18.390465+0000","last_became_active":"2023-11-24T06:17:17.421715+0000","last_became_peered":"2023-11-24T06:17:17.421715+0000","last_unstale":"2023-11-24T06:17:18.390465+0000","last_undegraded":"2023-11-24T06:17:18.390465+0000","last_fullsized":"2023-11-24T06:17:18.390465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:53.702868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.393086+0000","last_change":"2023-11-24T06:17:17.412252+0000","last_active":"2023-11-24T06:17:18.393086+0000","last_peered":"2023-11-24T06:17:18.393086+0000","last_clean":"2023-11-24T06:17:18.393086+0000","last_became_active":"2023-11-24T06:17:17.411979+0000","last_became_peered":"2023-11-24T06:17:17.411979+0000","last_unstale":"2023-11-24T06:17:18.393086+0000","last_undegraded":"2023-11-24T06:17:18.393086+0000","last_fullsized":"2023-11-24T06:17:18.393086+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:53.702868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820869+0000","last_change":"2023-11-24T06:17:17.425430+0000","last_active":"2023-11-24T06:17:18.820869+0000","last_peered":"2023-11-24T06:17:18.820869+0000","last_clean":"2023-11-24T06:17:18.820869+0000","last_became_active":"2023-11-24T06:17:17.424546+0000","last_became_peered":"2023-11-24T06:17:17.424546+0000","last_unstale":"2023-11-24T06:17:18.820869+0000","last_undegraded":"2023-11-24T06:17:18.820869+0000","last_fullsized":"2023-11-24T06:17:18.820869+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:01.018319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.394026+0000","last_change":"2023-11-24T06:17:17.427370+0000","last_active":"2023-11-24T06:17:18.394026+0000","last_peered":"2023-11-24T06:17:18.394026+0000","last_clean":"2023-11-24T06:17:18.394026+0000","last_became_active":"2023-11-24T06:17:17.426906+0000","last_became_peered":"2023-11-24T06:17:17.426906+0000","last_unstale":"2023-11-24T06:17:18.394026+0000","last_undegraded":"2023-11-24T06:17:18.394026+0000","last_fullsized":"2023-11-24T06:17:18.394026+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:18.330889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.821818+0000","last_change":"2023-11-24T06:17:17.417524+0000","last_active":"2023-11-24T06:17:18.821818+0000","last_peered":"2023-11-24T06:17:18.821818+0000","last_clean":"2023-11-24T06:17:18.821818+0000","last_became_active":"2023-11-24T06:17:17.417254+0000","last_became_peered":"2023-11-24T06:17:17.417254+0000","last_unstale":"2023-11-24T06:17:18.821818+0000","last_undegraded":"2023-11-24T06:17:18.821818+0000","last_fullsized":"2023-11-24T06:17:18.821818+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:08:42.324430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.816669+0000","last_change":"2023-11-24T06:17:17.426552+0000","last_active":"2023-11-24T06:17:18.816669+0000","last_peered":"2023-11-24T06:17:18.816669+0000","last_clean":"2023-11-24T06:17:18.816669+0000","last_became_active":"2023-11-24T06:17:17.426239+0000","last_became_peered":"2023-11-24T06:17:17.426239+0000","last_unstale":"2023-11-24T06:17:18.816669+0000","last_undegraded":"2023-11-24T06:17:18.816669+0000","last_fullsized":"2023-11-24T06:17:18.816669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:14:16.766695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387526+0000","last_change":"2023-11-24T06:17:17.425345+0000","last_active":"2023-11-24T06:17:18.387526+0000","last_peered":"2023-11-24T06:17:18.387526+0000","last_clean":"2023-11-24T06:17:18.387526+0000","last_became_active":"2023-11-24T06:17:17.425041+0000","last_became_peered":"2023-11-24T06:17:17.425041+0000","last_unstale":"2023-11-24T06:17:18.387526+0000","last_undegraded":"2023-11-24T06:17:18.387526+0000","last_fullsized":"2023-11-24T06:17:18.387526+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:04.241501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819585+0000","last_change":"2023-11-24T06:17:17.415267+0000","last_active":"2023-11-24T06:17:18.819585+0000","last_peered":"2023-11-24T06:17:18.819585+0000","last_clean":"2023-11-24T06:17:18.819585+0000","last_became_active":"2023-11-24T06:17:17.414973+0000","last_became_peered":"2023-11-24T06:17:17.414973+0000","last_unstale":"2023-11-24T06:17:18.819585+0000","last_undegraded":"2023-11-24T06:17:18.819585+0000","last_fullsized":"2023-11-24T06:17:18.819585+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:07.850355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391529+0000","last_change":"2023-11-24T06:17:17.419442+0000","last_active":"2023-11-24T06:17:18.391529+0000","last_peered":"2023-11-24T06:17:18.391529+0000","last_clean":"2023-11-24T06:17:18.391529+0000","last_became_active":"2023-11-24T06:17:17.419137+0000","last_became_peered":"2023-11-24T06:17:17.419137+0000","last_unstale":"2023-11-24T06:17:18.391529+0000","last_undegraded":"2023-11-24T06:17:18.391529+0000","last_fullsized":"2023-11-24T06:17:18.391529+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:01.018319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.394121+0000","last_change":"2023-11-24T06:17:17.425692+0000","last_active":"2023-11-24T06:17:18.394121+0000","last_peered":"2023-11-24T06:17:18.394121+0000","last_clean":"2023-11-24T06:17:18.394121+0000","last_became_active":"2023-11-24T06:17:17.422123+0000","last_became_peered":"2023-11-24T06:17:17.422123+0000","last_unstale":"2023-11-24T06:17:18.394121+0000","last_undegraded":"2023-11-24T06:17:18.394121+0000","last_fullsized":"2023-11-24T06:17:18.394121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:00.817489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.388264+0000","last_change":"2023-11-24T06:17:17.414817+0000","last_active":"2023-11-24T06:17:18.388264+0000","last_peered":"2023-11-24T06:17:18.388264+0000","last_clean":"2023-11-24T06:17:18.388264+0000","last_became_active":"2023-11-24T06:17:17.410953+0000","last_became_peered":"2023-11-24T06:17:17.410953+0000","last_unstale":"2023-11-24T06:17:18.388264+0000","last_undegraded":"2023-11-24T06:17:18.388264+0000","last_fullsized":"2023-11-24T06:17:18.388264+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:45.003151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387679+0000","last_change":"2023-11-24T06:17:17.421106+0000","last_active":"2023-11-24T06:17:18.387679+0000","last_peered":"2023-11-24T06:17:18.387679+0000","last_clean":"2023-11-24T06:17:18.387679+0000","last_became_active":"2023-11-24T06:17:17.420722+0000","last_became_peered":"2023-11-24T06:17:17.420722+0000","last_unstale":"2023-11-24T06:17:18.387679+0000","last_undegraded":"2023-11-24T06:17:18.387679+0000","last_fullsized":"2023-11-24T06:17:18.387679+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:18.330889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.817173+0000","last_change":"2023-11-24T06:17:17.422038+0000","last_active":"2023-11-24T06:17:18.817173+0000","last_peered":"2023-11-24T06:17:18.817173+0000","last_clean":"2023-11-24T06:17:18.817173+0000","last_became_active":"2023-11-24T06:17:17.421751+0000","last_became_peered":"2023-11-24T06:17:17.421751+0000","last_unstale":"2023-11-24T06:17:18.817173+0000","last_undegraded":"2023-11-24T06:17:18.817173+0000","last_fullsized":"2023-11-24T06:17:18.817173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:04:51.481600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.393087+0000","last_change":"2023-11-24T06:17:17.411841+0000","last_active":"2023-11-24T06:17:18.393087+0000","last_peered":"2023-11-24T06:17:18.393087+0000","last_clean":"2023-11-24T06:17:18.393087+0000","last_became_active":"2023-11-24T06:17:17.410346+0000","last_became_peered":"2023-11-24T06:17:17.410346+0000","last_unstale":"2023-11-24T06:17:18.393087+0000","last_undegraded":"2023-11-24T06:17:18.393087+0000","last_fullsized":"2023-11-24T06:17:18.393087+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:58.239341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391983+0000","last_change":"2023-11-24T06:17:17.409418+0000","last_active":"2023-11-24T06:17:18.391983+0000","last_peered":"2023-11-24T06:17:18.391983+0000","last_clean":"2023-11-24T06:17:18.391983+0000","last_became_active":"2023-11-24T06:17:17.408278+0000","last_became_peered":"2023-11-24T06:17:17.408278+0000","last_unstale":"2023-11-24T06:17:18.391983+0000","last_undegraded":"2023-11-24T06:17:18.391983+0000","last_fullsized":"2023-11-24T06:17:18.391983+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:56.627061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.812709+0000","last_change":"2023-11-24T06:17:17.419339+0000","last_active":"2023-11-24T06:17:18.812709+0000","last_peered":"2023-11-24T06:17:18.812709+0000","last_clean":"2023-11-24T06:17:18.812709+0000","last_became_active":"2023-11-24T06:17:17.418936+0000","last_became_peered":"2023-11-24T06:17:17.418936+0000","last_unstale":"2023-11-24T06:17:18.812709+0000","last_undegraded":"2023-11-24T06:17:18.812709+0000","last_fullsized":"2023-11-24T06:17:18.812709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:25:15.440363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.818943+0000","last_change":"2023-11-24T06:17:17.415059+0000","last_active":"2023-11-24T06:17:18.818943+0000","last_peered":"2023-11-24T06:17:18.818943+0000","last_clean":"2023-11-24T06:17:18.818943+0000","last_became_active":"2023-11-24T06:17:17.414713+0000","last_became_peered":"2023-11-24T06:17:17.414713+0000","last_unstale":"2023-11-24T06:17:18.818943+0000","last_undegraded":"2023-11-24T06:17:18.818943+0000","last_fullsized":"2023-11-24T06:17:18.818943+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:48.589643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820521+0000","last_change":"2023-11-24T06:17:17.420165+0000","last_active":"2023-11-24T06:17:18.820521+0000","last_peered":"2023-11-24T06:17:18.820521+0000","last_clean":"2023-11-24T06:17:18.820521+0000","last_became_active":"2023-11-24T06:17:17.419860+0000","last_became_peered":"2023-11-24T06:17:17.419860+0000","last_unstale":"2023-11-24T06:17:18.820521+0000","last_undegraded":"2023-11-24T06:17:18.820521+0000","last_fullsized":"2023-11-24T06:17:18.820521+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:23:58.898486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.388372+0000","last_change":"2023-11-24T06:17:17.422622+0000","last_active":"2023-11-24T06:17:18.388372+0000","last_peered":"2023-11-24T06:17:18.388372+0000","last_clean":"2023-11-24T06:17:18.388372+0000","last_became_active":"2023-11-24T06:17:17.420794+0000","last_became_peered":"2023-11-24T06:17:17.420794+0000","last_unstale":"2023-11-24T06:17:18.388372+0000","last_undegraded":"2023-11-24T06:17:18.388372+0000","last_fullsized":"2023-11-24T06:17:18.388372+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:04.564962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820115+0000","last_change":"2023-11-24T06:17:17.417600+0000","last_active":"2023-11-24T06:17:18.820115+0000","last_peered":"2023-11-24T06:17:18.820115+0000","last_clean":"2023-11-24T06:17:18.820115+0000","last_became_active":"2023-11-24T06:17:17.413146+0000","last_became_peered":"2023-11-24T06:17:17.413146+0000","last_unstale":"2023-11-24T06:17:18.820115+0000","last_undegraded":"2023-11-24T06:17:18.820115+0000","last_fullsized":"2023-11-24T06:17:18.820115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:58.239341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.821699+0000","last_change":"2023-11-24T06:17:17.409350+0000","last_active":"2023-11-24T06:17:18.821699+0000","last_peered":"2023-11-24T06:17:18.821699+0000","last_clean":"2023-11-24T06:17:18.821699+0000","last_became_active":"2023-11-24T06:17:17.409064+0000","last_became_peered":"2023-11-24T06:17:17.409064+0000","last_unstale":"2023-11-24T06:17:18.821699+0000","last_undegraded":"2023-11-24T06:17:18.821699+0000","last_fullsized":"2023-11-24T06:17:18.821699+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:48.982744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392233+0000","last_change":"2023-11-24T06:17:17.419291+0000","last_active":"2023-11-24T06:17:18.392233+0000","last_peered":"2023-11-24T06:17:18.392233+0000","last_clean":"2023-11-24T06:17:18.392233+0000","last_became_active":"2023-11-24T06:17:17.418942+0000","last_became_peered":"2023-11-24T06:17:17.418942+0000","last_unstale":"2023-11-24T06:17:18.392233+0000","last_undegraded":"2023-11-24T06:17:18.392233+0000","last_fullsized":"2023-11-24T06:17:18.392233+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:12.852430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.393183+0000","last_change":"2023-11-24T06:17:17.418613+0000","last_active":"2023-11-24T06:17:18.393183+0000","last_peered":"2023-11-24T06:17:18.393183+0000","last_clean":"2023-11-24T06:17:18.393183+0000","last_became_active":"2023-11-24T06:17:17.417943+0000","last_became_peered":"2023-11-24T06:17:17.417943+0000","last_unstale":"2023-11-24T06:17:18.393183+0000","last_undegraded":"2023-11-24T06:17:18.393183+0000","last_fullsized":"2023-11-24T06:17:18.393183+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:04.241501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819895+0000","last_change":"2023-11-24T06:17:17.425415+0000","last_active":"2023-11-24T06:17:18.819895+0000","last_peered":"2023-11-24T06:17:18.819895+0000","last_clean":"2023-11-24T06:17:18.819895+0000","last_became_active":"2023-11-24T06:17:17.424917+0000","last_became_peered":"2023-11-24T06:17:17.424917+0000","last_unstale":"2023-11-24T06:17:18.819895+0000","last_undegraded":"2023-11-24T06:17:18.819895+0000","last_fullsized":"2023-11-24T06:17:18.819895+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:14.150534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.390466+0000","last_change":"2023-11-24T06:17:17.418861+0000","last_active":"2023-11-24T06:17:18.390466+0000","last_peered":"2023-11-24T06:17:18.390466+0000","last_clean":"2023-11-24T06:17:18.390466+0000","last_became_active":"2023-11-24T06:17:17.418279+0000","last_became_peered":"2023-11-24T06:17:17.418279+0000","last_unstale":"2023-11-24T06:17:18.390466+0000","last_undegraded":"2023-11-24T06:17:18.390466+0000","last_fullsized":"2023-11-24T06:17:18.390466+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:58.239341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.814567+0000","last_change":"2023-11-24T06:17:17.419892+0000","last_active":"2023-11-24T06:17:18.814567+0000","last_peered":"2023-11-24T06:17:18.814567+0000","last_clean":"2023-11-24T06:17:18.814567+0000","last_became_active":"2023-11-24T06:17:17.419612+0000","last_became_peered":"2023-11-24T06:17:17.419612+0000","last_unstale":"2023-11-24T06:17:18.814567+0000","last_undegraded":"2023-11-24T06:17:18.814567+0000","last_fullsized":"2023-11-24T06:17:18.814567+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:33.824783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820695+0000","last_change":"2023-11-24T06:17:16.425069+0000","last_active":"2023-11-24T06:17:18.820695+0000","last_peered":"2023-11-24T06:17:18.820695+0000","last_clean":"2023-11-24T06:17:18.820695+0000","last_became_active":"2023-11-24T06:17:16.424500+0000","last_became_peered":"2023-11-24T06:17:16.424500+0000","last_unstale":"2023-11-24T06:17:18.820695+0000","last_undegraded":"2023-11-24T06:17:18.820695+0000","last_fullsized":"2023-11-24T06:17:18.820695+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:29.090827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820809+0000","last_change":"2023-11-24T06:17:17.414213+0000","last_active":"2023-11-24T06:17:18.820809+0000","last_peered":"2023-11-24T06:17:18.820809+0000","last_clean":"2023-11-24T06:17:18.820809+0000","last_became_active":"2023-11-24T06:17:17.412662+0000","last_became_peered":"2023-11-24T06:17:17.412662+0000","last_unstale":"2023-11-24T06:17:18.820809+0000","last_undegraded":"2023-11-24T06:17:18.820809+0000","last_fullsized":"2023-11-24T06:17:18.820809+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:25:15.440363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.814815+0000","last_change":"2023-11-24T06:17:16.430731+0000","last_active":"2023-11-24T06:17:18.814815+0000","last_peered":"2023-11-24T06:17:18.814815+0000","last_clean":"2023-11-24T06:17:18.814815+0000","last_became_active":"2023-11-24T06:17:16.430185+0000","last_became_peered":"2023-11-24T06:17:16.430185+0000","last_unstale":"2023-11-24T06:17:18.814815+0000","last_undegraded":"2023-11-24T06:17:18.814815+0000","last_fullsized":"2023-11-24T06:17:18.814815+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:43:43.843622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392160+0000","last_change":"2023-11-24T06:17:17.416794+0000","last_active":"2023-11-24T06:17:18.392160+0000","last_peered":"2023-11-24T06:17:18.392160+0000","last_clean":"2023-11-24T06:17:18.392160+0000","last_became_active":"2023-11-24T06:17:17.416449+0000","last_became_peered":"2023-11-24T06:17:17.416449+0000","last_unstale":"2023-11-24T06:17:18.392160+0000","last_undegraded":"2023-11-24T06:17:18.392160+0000","last_fullsized":"2023-11-24T06:17:18.392160+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:24:04.806063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387692+0000","last_change":"2023-11-24T06:17:16.416080+0000","last_active":"2023-11-24T06:17:18.387692+0000","last_peered":"2023-11-24T06:17:18.387692+0000","last_clean":"2023-11-24T06:17:18.387692+0000","last_became_active":"2023-11-24T06:17:16.415751+0000","last_became_peered":"2023-11-24T06:17:16.415751+0000","last_unstale":"2023-11-24T06:17:18.387692+0000","last_undegraded":"2023-11-24T06:17:18.387692+0000","last_fullsized":"2023-11-24T06:17:18.387692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:00.002770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820309+0000","last_change":"2023-11-24T06:17:17.415686+0000","last_active":"2023-11-24T06:17:18.820309+0000","last_peered":"2023-11-24T06:17:18.820309+0000","last_clean":"2023-11-24T06:17:18.820309+0000","last_became_active":"2023-11-24T06:17:17.413606+0000","last_became_peered":"2023-11-24T06:17:17.413606+0000","last_unstale":"2023-11-24T06:17:18.820309+0000","last_undegraded":"2023-11-24T06:17:18.820309+0000","last_fullsized":"2023-11-24T06:17:18.820309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:53.702868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.817166+0000","last_change":"2023-11-24T06:17:16.427095+0000","last_active":"2023-11-24T06:17:18.817166+0000","last_peered":"2023-11-24T06:17:18.817166+0000","last_clean":"2023-11-24T06:17:18.817166+0000","last_became_active":"2023-11-24T06:17:16.425462+0000","last_became_peered":"2023-11-24T06:17:16.425462+0000","last_unstale":"2023-11-24T06:17:18.817166+0000","last_undegraded":"2023-11-24T06:17:18.817166+0000","last_fullsized":"2023-11-24T06:17:18.817166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:23:42.085007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.393288+0000","last_change":"2023-11-24T06:17:17.418512+0000","last_active":"2023-11-24T06:17:18.393288+0000","last_peered":"2023-11-24T06:17:18.393288+0000","last_clean":"2023-11-24T06:17:18.393288+0000","last_became_active":"2023-11-24T06:17:17.418120+0000","last_became_peered":"2023-11-24T06:17:17.418120+0000","last_unstale":"2023-11-24T06:17:18.393288+0000","last_undegraded":"2023-11-24T06:17:18.393288+0000","last_fullsized":"2023-11-24T06:17:18.393288+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:28.122398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392516+0000","last_change":"2023-11-24T06:17:16.425516+0000","last_active":"2023-11-24T06:17:18.392516+0000","last_peered":"2023-11-24T06:17:18.392516+0000","last_clean":"2023-11-24T06:17:18.392516+0000","last_became_active":"2023-11-24T06:17:16.424425+0000","last_became_peered":"2023-11-24T06:17:16.424425+0000","last_unstale":"2023-11-24T06:17:18.392516+0000","last_undegraded":"2023-11-24T06:17:18.392516+0000","last_fullsized":"2023-11-24T06:17:18.392516+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:33:29.653709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.821224+0000","last_change":"2023-11-24T06:17:17.409205+0000","last_active":"2023-11-24T06:17:18.821224+0000","last_peered":"2023-11-24T06:17:18.821224+0000","last_clean":"2023-11-24T06:17:18.821224+0000","last_became_active":"2023-11-24T06:17:17.408937+0000","last_became_peered":"2023-11-24T06:17:17.408937+0000","last_unstale":"2023-11-24T06:17:18.821224+0000","last_undegraded":"2023-11-24T06:17:18.821224+0000","last_fullsized":"2023-11-24T06:17:18.821224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:45:54.332599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391852+0000","last_change":"2023-11-24T06:17:16.424793+0000","last_active":"2023-11-24T06:17:18.391852+0000","last_peered":"2023-11-24T06:17:18.391852+0000","last_clean":"2023-11-24T06:17:18.391852+0000","last_became_active":"2023-11-24T06:17:16.423487+0000","last_became_peered":"2023-11-24T06:17:16.423487+0000","last_unstale":"2023-11-24T06:17:18.391852+0000","last_undegraded":"2023-11-24T06:17:18.391852+0000","last_fullsized":"2023-11-24T06:17:18.391852+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:53.398555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391212+0000","last_change":"2023-11-24T06:17:17.410932+0000","last_active":"2023-11-24T06:17:18.391212+0000","last_peered":"2023-11-24T06:17:18.391212+0000","last_clean":"2023-11-24T06:17:18.391212+0000","last_became_active":"2023-11-24T06:17:17.410624+0000","last_became_peered":"2023-11-24T06:17:17.410624+0000","last_unstale":"2023-11-24T06:17:18.391212+0000","last_undegraded":"2023-11-24T06:17:18.391212+0000","last_fullsized":"2023-11-24T06:17:18.391212+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:37:15.493664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387286+0000","last_change":"2023-11-24T06:17:16.414388+0000","last_active":"2023-11-24T06:17:18.387286+0000","last_peered":"2023-11-24T06:17:18.387286+0000","last_clean":"2023-11-24T06:17:18.387286+0000","last_became_active":"2023-11-24T06:17:16.414096+0000","last_became_peered":"2023-11-24T06:17:16.414096+0000","last_unstale":"2023-11-24T06:17:18.387286+0000","last_undegraded":"2023-11-24T06:17:18.387286+0000","last_fullsized":"2023-11-24T06:17:18.387286+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:52.687319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819361+0000","last_change":"2023-11-24T06:17:17.424413+0000","last_active":"2023-11-24T06:17:18.819361+0000","last_peered":"2023-11-24T06:17:18.819361+0000","last_clean":"2023-11-24T06:17:18.819361+0000","last_became_active":"2023-11-24T06:17:17.424105+0000","last_became_peered":"2023-11-24T06:17:17.424105+0000","last_unstale":"2023-11-24T06:17:18.819361+0000","last_undegraded":"2023-11-24T06:17:18.819361+0000","last_fullsized":"2023-11-24T06:17:18.819361+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:56.627061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391140+0000","last_change":"2023-11-24T06:17:16.425492+0000","last_active":"2023-11-24T06:17:18.391140+0000","last_peered":"2023-11-24T06:17:18.391140+0000","last_clean":"2023-11-24T06:17:18.391140+0000","last_became_active":"2023-11-24T06:17:16.424454+0000","last_became_peered":"2023-11-24T06:17:16.424454+0000","last_unstale":"2023-11-24T06:17:18.391140+0000","last_undegraded":"2023-11-24T06:17:18.391140+0000","last_fullsized":"2023-11-24T06:17:18.391140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:26:39.607586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.821910+0000","last_change":"2023-11-24T06:17:17.420252+0000","last_active":"2023-11-24T06:17:18.821910+0000","last_peered":"2023-11-24T06:17:18.821910+0000","last_clean":"2023-11-24T06:17:18.821910+0000","last_became_active":"2023-11-24T06:17:17.419989+0000","last_became_peered":"2023-11-24T06:17:17.419989+0000","last_unstale":"2023-11-24T06:17:18.821910+0000","last_undegraded":"2023-11-24T06:17:18.821910+0000","last_fullsized":"2023-11-24T06:17:18.821910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:13:24.090710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387881+0000","last_change":"2023-11-24T06:17:16.429657+0000","last_active":"2023-11-24T06:17:18.387881+0000","last_peered":"2023-11-24T06:17:18.387881+0000","last_clean":"2023-11-24T06:17:18.387881+0000","last_became_active":"2023-11-24T06:17:16.429400+0000","last_became_peered":"2023-11-24T06:17:16.429400+0000","last_unstale":"2023-11-24T06:17:18.387881+0000","last_undegraded":"2023-11-24T06:17:18.387881+0000","last_fullsized":"2023-11-24T06:17:18.387881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:52.900957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387948+0000","last_change":"2023-11-24T06:17:17.416803+0000","last_active":"2023-11-24T06:17:18.387948+0000","last_peered":"2023-11-24T06:17:18.387948+0000","last_clean":"2023-11-24T06:17:18.387948+0000","last_became_active":"2023-11-24T06:17:17.416429+0000","last_became_peered":"2023-11-24T06:17:17.416429+0000","last_unstale":"2023-11-24T06:17:18.387948+0000","last_undegraded":"2023-11-24T06:17:18.387948+0000","last_fullsized":"2023-11-24T06:17:18.387948+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:27:55.321099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.814778+0000","last_change":"2023-11-24T06:17:16.409047+0000","last_active":"2023-11-24T06:17:18.814778+0000","last_peered":"2023-11-24T06:17:18.814778+0000","last_clean":"2023-11-24T06:17:18.814778+0000","last_became_active":"2023-11-24T06:17:16.408751+0000","last_became_peered":"2023-11-24T06:17:16.408751+0000","last_unstale":"2023-11-24T06:17:18.814778+0000","last_undegraded":"2023-11-24T06:17:18.814778+0000","last_fullsized":"2023-11-24T06:17:18.814778+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:13.134985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.388177+0000","last_change":"2023-11-24T06:17:17.422892+0000","last_active":"2023-11-24T06:17:18.388177+0000","last_peered":"2023-11-24T06:17:18.388177+0000","last_clean":"2023-11-24T06:17:18.388177+0000","last_became_active":"2023-11-24T06:17:17.422339+0000","last_became_peered":"2023-11-24T06:17:17.422339+0000","last_unstale":"2023-11-24T06:17:18.388177+0000","last_undegraded":"2023-11-24T06:17:18.388177+0000","last_fullsized":"2023-11-24T06:17:18.388177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:39.811108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.822149+0000","last_change":"2023-11-24T06:17:16.431179+0000","last_active":"2023-11-24T06:17:18.822149+0000","last_peered":"2023-11-24T06:17:18.822149+0000","last_clean":"2023-11-24T06:17:18.822149+0000","last_became_active":"2023-11-24T06:17:16.430031+0000","last_became_peered":"2023-11-24T06:17:16.430031+0000","last_unstale":"2023-11-24T06:17:18.822149+0000","last_undegraded":"2023-11-24T06:17:18.822149+0000","last_fullsized":"2023-11-24T06:17:18.822149+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:00.578190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.821841+0000","last_change":"2023-11-24T06:17:17.422022+0000","last_active":"2023-11-24T06:17:18.821841+0000","last_peered":"2023-11-24T06:17:18.821841+0000","last_clean":"2023-11-24T06:17:18.821841+0000","last_became_active":"2023-11-24T06:17:17.421753+0000","last_became_peered":"2023-11-24T06:17:17.421753+0000","last_unstale":"2023-11-24T06:17:18.821841+0000","last_undegraded":"2023-11-24T06:17:18.821841+0000","last_fullsized":"2023-11-24T06:17:18.821841+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:24:28.186658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.388092+0000","last_change":"2023-11-24T06:17:16.428348+0000","last_active":"2023-11-24T06:17:18.388092+0000","last_peered":"2023-11-24T06:17:18.388092+0000","last_clean":"2023-11-24T06:17:18.388092+0000","last_became_active":"2023-11-24T06:17:16.426800+0000","last_became_peered":"2023-11-24T06:17:16.426800+0000","last_unstale":"2023-11-24T06:17:18.388092+0000","last_undegraded":"2023-11-24T06:17:18.388092+0000","last_fullsized":"2023-11-24T06:17:18.388092+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:23:57.882937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819403+0000","last_change":"2023-11-24T06:17:17.423622+0000","last_active":"2023-11-24T06:17:18.819403+0000","last_peered":"2023-11-24T06:17:18.819403+0000","last_clean":"2023-11-24T06:17:18.819403+0000","last_became_active":"2023-11-24T06:17:17.421139+0000","last_became_peered":"2023-11-24T06:17:17.421139+0000","last_unstale":"2023-11-24T06:17:18.819403+0000","last_undegraded":"2023-11-24T06:17:18.819403+0000","last_fullsized":"2023-11-24T06:17:18.819403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:24:04.806063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392777+0000","last_change":"2023-11-24T06:17:16.419010+0000","last_active":"2023-11-24T06:17:18.392777+0000","last_peered":"2023-11-24T06:17:18.392777+0000","last_clean":"2023-11-24T06:17:18.392777+0000","last_became_active":"2023-11-24T06:17:16.418637+0000","last_became_peered":"2023-11-24T06:17:16.418637+0000","last_unstale":"2023-11-24T06:17:18.392777+0000","last_undegraded":"2023-11-24T06:17:18.392777+0000","last_fullsized":"2023-11-24T06:17:18.392777+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:34.496015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387745+0000","last_change":"2023-11-24T06:17:17.416817+0000","last_active":"2023-11-24T06:17:18.387745+0000","last_peered":"2023-11-24T06:17:18.387745+0000","last_clean":"2023-11-24T06:17:18.387745+0000","last_became_active":"2023-11-24T06:17:17.415352+0000","last_became_peered":"2023-11-24T06:17:17.415352+0000","last_unstale":"2023-11-24T06:17:18.387745+0000","last_undegraded":"2023-11-24T06:17:18.387745+0000","last_fullsized":"2023-11-24T06:17:18.387745+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:00.817489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819350+0000","last_change":"2023-11-24T06:17:16.423325+0000","last_active":"2023-11-24T06:17:18.819350+0000","last_peered":"2023-11-24T06:17:18.819350+0000","last_clean":"2023-11-24T06:17:18.819350+0000","last_became_active":"2023-11-24T06:17:16.423000+0000","last_became_peered":"2023-11-24T06:17:16.423000+0000","last_unstale":"2023-11-24T06:17:18.819350+0000","last_undegraded":"2023-11-24T06:17:18.819350+0000","last_fullsized":"2023-11-24T06:17:18.819350+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:34.496015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.390839+0000","last_change":"2023-11-24T06:17:17.418624+0000","last_active":"2023-11-24T06:17:18.390839+0000","last_peered":"2023-11-24T06:17:18.390839+0000","last_clean":"2023-11-24T06:17:18.390839+0000","last_became_active":"2023-11-24T06:17:17.418236+0000","last_became_peered":"2023-11-24T06:17:17.418236+0000","last_unstale":"2023-11-24T06:17:18.390839+0000","last_undegraded":"2023-11-24T06:17:18.390839+0000","last_fullsized":"2023-11-24T06:17:18.390839+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:23:58.898486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387160+0000","last_change":"2023-11-24T06:17:16.429252+0000","last_active":"2023-11-24T06:17:18.387160+0000","last_peered":"2023-11-24T06:17:18.387160+0000","last_clean":"2023-11-24T06:17:18.387160+0000","last_became_active":"2023-11-24T06:17:16.428863+0000","last_became_peered":"2023-11-24T06:17:16.428863+0000","last_unstale":"2023-11-24T06:17:18.387160+0000","last_undegraded":"2023-11-24T06:17:18.387160+0000","last_fullsized":"2023-11-24T06:17:18.387160+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:04:50.451978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820255+0000","last_change":"2023-11-24T06:17:17.415156+0000","last_active":"2023-11-24T06:17:18.820255+0000","last_peered":"2023-11-24T06:17:18.820255+0000","last_clean":"2023-11-24T06:17:18.820255+0000","last_became_active":"2023-11-24T06:17:17.414507+0000","last_became_peered":"2023-11-24T06:17:17.414507+0000","last_unstale":"2023-11-24T06:17:18.820255+0000","last_undegraded":"2023-11-24T06:17:18.820255+0000","last_fullsized":"2023-11-24T06:17:18.820255+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:53:16.931385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387783+0000","last_change":"2023-11-24T06:17:16.420651+0000","last_active":"2023-11-24T06:17:18.387783+0000","last_peered":"2023-11-24T06:17:18.387783+0000","last_clean":"2023-11-24T06:17:18.387783+0000","last_became_active":"2023-11-24T06:17:16.420041+0000","last_became_peered":"2023-11-24T06:17:16.420041+0000","last_unstale":"2023-11-24T06:17:18.387783+0000","last_undegraded":"2023-11-24T06:17:18.387783+0000","last_fullsized":"2023-11-24T06:17:18.387783+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:57.223792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392669+0000","last_change":"2023-11-24T06:17:17.417057+0000","last_active":"2023-11-24T06:17:18.392669+0000","last_peered":"2023-11-24T06:17:18.392669+0000","last_clean":"2023-11-24T06:17:18.392669+0000","last_became_active":"2023-11-24T06:17:17.416064+0000","last_became_peered":"2023-11-24T06:17:17.416064+0000","last_unstale":"2023-11-24T06:17:18.392669+0000","last_undegraded":"2023-11-24T06:17:18.392669+0000","last_fullsized":"2023-11-24T06:17:18.392669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:04:51.467527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820659+0000","last_change":"2023-11-24T06:17:16.420014+0000","last_active":"2023-11-24T06:17:18.820659+0000","last_peered":"2023-11-24T06:17:18.820659+0000","last_clean":"2023-11-24T06:17:18.820659+0000","last_became_active":"2023-11-24T06:17:16.419682+0000","last_became_peered":"2023-11-24T06:17:16.419682+0000","last_unstale":"2023-11-24T06:17:18.820659+0000","last_undegraded":"2023-11-24T06:17:18.820659+0000","last_fullsized":"2023-11-24T06:17:18.820659+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:50:20.964101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.822727+0000","last_change":"2023-11-24T06:17:17.413988+0000","last_active":"2023-11-24T06:17:18.822727+0000","last_peered":"2023-11-24T06:17:18.822727+0000","last_clean":"2023-11-24T06:17:18.822727+0000","last_became_active":"2023-11-24T06:17:17.413634+0000","last_became_peered":"2023-11-24T06:17:17.413634+0000","last_unstale":"2023-11-24T06:17:18.822727+0000","last_undegraded":"2023-11-24T06:17:18.822727+0000","last_fullsized":"2023-11-24T06:17:18.822727+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:31.519613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392673+0000","last_change":"2023-11-24T06:17:16.418846+0000","last_active":"2023-11-24T06:17:18.392673+0000","last_peered":"2023-11-24T06:17:18.392673+0000","last_clean":"2023-11-24T06:17:18.392673+0000","last_became_active":"2023-11-24T06:17:16.418582+0000","last_became_peered":"2023-11-24T06:17:16.418582+0000","last_unstale":"2023-11-24T06:17:18.392673+0000","last_undegraded":"2023-11-24T06:17:18.392673+0000","last_fullsized":"2023-11-24T06:17:18.392673+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:56:50.498549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820014+0000","last_change":"2023-11-24T06:17:17.413669+0000","last_active":"2023-11-24T06:17:18.820014+0000","last_peered":"2023-11-24T06:17:18.820014+0000","last_clean":"2023-11-24T06:17:18.820014+0000","last_became_active":"2023-11-24T06:17:17.413014+0000","last_became_peered":"2023-11-24T06:17:17.413014+0000","last_unstale":"2023-11-24T06:17:18.820014+0000","last_undegraded":"2023-11-24T06:17:18.820014+0000","last_fullsized":"2023-11-24T06:17:18.820014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:04:51.467527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392341+0000","last_change":"2023-11-24T06:17:16.421306+0000","last_active":"2023-11-24T06:17:18.392341+0000","last_peered":"2023-11-24T06:17:18.392341+0000","last_clean":"2023-11-24T06:17:18.392341+0000","last_became_active":"2023-11-24T06:17:16.420998+0000","last_became_peered":"2023-11-24T06:17:16.420998+0000","last_unstale":"2023-11-24T06:17:18.392341+0000","last_undegraded":"2023-11-24T06:17:18.392341+0000","last_fullsized":"2023-11-24T06:17:18.392341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:07.073655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.816827+0000","last_change":"2023-11-24T06:17:17.415530+0000","last_active":"2023-11-24T06:17:18.816827+0000","last_peered":"2023-11-24T06:17:18.816827+0000","last_clean":"2023-11-24T06:17:18.816827+0000","last_became_active":"2023-11-24T06:17:17.415239+0000","last_became_peered":"2023-11-24T06:17:17.415239+0000","last_unstale":"2023-11-24T06:17:18.816827+0000","last_undegraded":"2023-11-24T06:17:18.816827+0000","last_fullsized":"2023-11-24T06:17:18.816827+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:50.847709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392420+0000","last_change":"2023-11-24T06:17:16.421728+0000","last_active":"2023-11-24T06:17:18.392420+0000","last_peered":"2023-11-24T06:17:18.392420+0000","last_clean":"2023-11-24T06:17:18.392420+0000","last_became_active":"2023-11-24T06:17:16.421403+0000","last_became_peered":"2023-11-24T06:17:16.421403+0000","last_unstale":"2023-11-24T06:17:18.392420+0000","last_undegraded":"2023-11-24T06:17:18.392420+0000","last_fullsized":"2023-11-24T06:17:18.392420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:54:40.137540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.817559+0000","last_change":"2023-11-24T06:17:17.416153+0000","last_active":"2023-11-24T06:17:18.817559+0000","last_peered":"2023-11-24T06:17:18.817559+0000","last_clean":"2023-11-24T06:17:18.817559+0000","last_became_active":"2023-11-24T06:17:17.414853+0000","last_became_peered":"2023-11-24T06:17:17.414853+0000","last_unstale":"2023-11-24T06:17:18.817559+0000","last_undegraded":"2023-11-24T06:17:18.817559+0000","last_fullsized":"2023-11-24T06:17:18.817559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:43:04.857890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.813862+0000","last_change":"2023-11-24T06:17:16.422145+0000","last_active":"2023-11-24T06:17:18.813862+0000","last_peered":"2023-11-24T06:17:18.813862+0000","last_clean":"2023-11-24T06:17:18.813862+0000","last_became_active":"2023-11-24T06:17:16.421869+0000","last_became_peered":"2023-11-24T06:17:16.421869+0000","last_unstale":"2023-11-24T06:17:18.813862+0000","last_undegraded":"2023-11-24T06:17:18.813862+0000","last_fullsized":"2023-11-24T06:17:18.813862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:04:50.466051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.816870+0000","last_change":"2023-11-24T06:17:17.412904+0000","last_active":"2023-11-24T06:17:18.816870+0000","last_peered":"2023-11-24T06:17:18.816870+0000","last_clean":"2023-11-24T06:17:18.816870+0000","last_became_active":"2023-11-24T06:17:17.412575+0000","last_became_peered":"2023-11-24T06:17:17.412575+0000","last_unstale":"2023-11-24T06:17:18.816870+0000","last_undegraded":"2023-11-24T06:17:18.816870+0000","last_fullsized":"2023-11-24T06:17:18.816870+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:50.227692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819989+0000","last_change":"2023-11-24T06:17:16.422042+0000","last_active":"2023-11-24T06:17:18.819989+0000","last_peered":"2023-11-24T06:17:18.819989+0000","last_clean":"2023-11-24T06:17:18.819989+0000","last_became_active":"2023-11-24T06:17:16.421002+0000","last_became_peered":"2023-11-24T06:17:16.421002+0000","last_unstale":"2023-11-24T06:17:18.819989+0000","last_undegraded":"2023-11-24T06:17:18.819989+0000","last_fullsized":"2023-11-24T06:17:18.819989+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:22.786967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.816510+0000","last_change":"2023-11-24T06:17:17.416486+0000","last_active":"2023-11-24T06:17:18.816510+0000","last_peered":"2023-11-24T06:17:18.816510+0000","last_clean":"2023-11-24T06:17:18.816510+0000","last_became_active":"2023-11-24T06:17:17.416143+0000","last_became_peered":"2023-11-24T06:17:17.416143+0000","last_unstale":"2023-11-24T06:17:18.816510+0000","last_undegraded":"2023-11-24T06:17:18.816510+0000","last_fullsized":"2023-11-24T06:17:18.816510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:36.175677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391162+0000","last_change":"2023-11-24T06:17:16.426462+0000","last_active":"2023-11-24T06:17:18.391162+0000","last_peered":"2023-11-24T06:17:18.391162+0000","last_clean":"2023-11-24T06:17:18.391162+0000","last_became_active":"2023-11-24T06:17:16.426184+0000","last_became_peered":"2023-11-24T06:17:16.426184+0000","last_unstale":"2023-11-24T06:17:18.391162+0000","last_undegraded":"2023-11-24T06:17:18.391162+0000","last_fullsized":"2023-11-24T06:17:18.391162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:05.040137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392080+0000","last_change":"2023-11-24T06:17:17.420511+0000","last_active":"2023-11-24T06:17:18.392080+0000","last_peered":"2023-11-24T06:17:18.392080+0000","last_clean":"2023-11-24T06:17:18.392080+0000","last_became_active":"2023-11-24T06:17:17.420133+0000","last_became_peered":"2023-11-24T06:17:17.420133+0000","last_unstale":"2023-11-24T06:17:18.392080+0000","last_undegraded":"2023-11-24T06:17:18.392080+0000","last_fullsized":"2023-11-24T06:17:18.392080+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:53.916506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.818785+0000","last_change":"2023-11-24T06:17:16.430044+0000","last_active":"2023-11-24T06:17:18.818785+0000","last_peered":"2023-11-24T06:17:18.818785+0000","last_clean":"2023-11-24T06:17:18.818785+0000","last_became_active":"2023-11-24T06:17:16.429766+0000","last_became_peered":"2023-11-24T06:17:16.429766+0000","last_unstale":"2023-11-24T06:17:18.818785+0000","last_undegraded":"2023-11-24T06:17:18.818785+0000","last_fullsized":"2023-11-24T06:17:18.818785+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:07:47.032762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.816590+0000","last_change":"2023-11-24T06:17:17.422038+0000","last_active":"2023-11-24T06:17:18.816590+0000","last_peered":"2023-11-24T06:17:18.816590+0000","last_clean":"2023-11-24T06:17:18.816590+0000","last_became_active":"2023-11-24T06:17:17.421727+0000","last_became_peered":"2023-11-24T06:17:17.421727+0000","last_unstale":"2023-11-24T06:17:18.816590+0000","last_undegraded":"2023-11-24T06:17:18.816590+0000","last_fullsized":"2023-11-24T06:17:18.816590+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:09.112474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820758+0000","last_change":"2023-11-24T06:17:16.427797+0000","last_active":"2023-11-24T06:17:18.820758+0000","last_peered":"2023-11-24T06:17:18.820758+0000","last_clean":"2023-11-24T06:17:18.820758+0000","last_became_active":"2023-11-24T06:17:16.427478+0000","last_became_peered":"2023-11-24T06:17:16.427478+0000","last_unstale":"2023-11-24T06:17:18.820758+0000","last_undegraded":"2023-11-24T06:17:18.820758+0000","last_fullsized":"2023-11-24T06:17:18.820758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:37:14.478115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.822726+0000","last_change":"2023-11-24T06:17:17.413936+0000","last_active":"2023-11-24T06:17:18.822726+0000","last_peered":"2023-11-24T06:17:18.822726+0000","last_clean":"2023-11-24T06:17:18.822726+0000","last_became_active":"2023-11-24T06:17:17.413529+0000","last_became_peered":"2023-11-24T06:17:17.413529+0000","last_unstale":"2023-11-24T06:17:18.822726+0000","last_undegraded":"2023-11-24T06:17:18.822726+0000","last_fullsized":"2023-11-24T06:17:18.822726+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:41.919440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.815004+0000","last_change":"2023-11-24T06:17:16.422261+0000","last_active":"2023-11-24T06:17:18.815004+0000","last_peered":"2023-11-24T06:17:18.815004+0000","last_clean":"2023-11-24T06:17:18.815004+0000","last_became_active":"2023-11-24T06:17:16.421875+0000","last_became_peered":"2023-11-24T06:17:16.421875+0000","last_unstale":"2023-11-24T06:17:18.815004+0000","last_undegraded":"2023-11-24T06:17:18.815004+0000","last_fullsized":"2023-11-24T06:17:18.815004+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:53:15.915836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.812708+0000","last_change":"2023-11-24T06:17:17.416143+0000","last_active":"2023-11-24T06:17:18.812708+0000","last_peered":"2023-11-24T06:17:18.812708+0000","last_clean":"2023-11-24T06:17:18.812708+0000","last_became_active":"2023-11-24T06:17:17.412289+0000","last_became_peered":"2023-11-24T06:17:17.412289+0000","last_unstale":"2023-11-24T06:17:18.812708+0000","last_undegraded":"2023-11-24T06:17:18.812708+0000","last_fullsized":"2023-11-24T06:17:18.812708+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:25.646482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820177+0000","last_change":"2023-11-24T06:17:16.421114+0000","last_active":"2023-11-24T06:17:18.820177+0000","last_peered":"2023-11-24T06:17:18.820177+0000","last_clean":"2023-11-24T06:17:18.820177+0000","last_became_active":"2023-11-24T06:17:16.419740+0000","last_became_peered":"2023-11-24T06:17:16.419740+0000","last_unstale":"2023-11-24T06:17:18.820177+0000","last_undegraded":"2023-11-24T06:17:18.820177+0000","last_fullsized":"2023-11-24T06:17:18.820177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:33:29.653709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819771+0000","last_change":"2023-11-24T06:17:17.425941+0000","last_active":"2023-11-24T06:17:18.819771+0000","last_peered":"2023-11-24T06:17:18.819771+0000","last_clean":"2023-11-24T06:17:18.819771+0000","last_became_active":"2023-11-24T06:17:17.421314+0000","last_became_peered":"2023-11-24T06:17:17.421314+0000","last_unstale":"2023-11-24T06:17:18.819771+0000","last_undegraded":"2023-11-24T06:17:18.819771+0000","last_fullsized":"2023-11-24T06:17:18.819771+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:43:44.859171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.814100+0000","last_change":"2023-11-24T06:17:16.421585+0000","last_active":"2023-11-24T06:17:18.814100+0000","last_peered":"2023-11-24T06:17:18.814100+0000","last_clean":"2023-11-24T06:17:18.814100+0000","last_became_active":"2023-11-24T06:17:16.420794+0000","last_became_peered":"2023-11-24T06:17:16.420794+0000","last_unstale":"2023-11-24T06:17:18.814100+0000","last_undegraded":"2023-11-24T06:17:18.814100+0000","last_fullsized":"2023-11-24T06:17:18.814100+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:43:03.842341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.813754+0000","last_change":"2023-11-24T06:17:17.413706+0000","last_active":"2023-11-24T06:17:18.813754+0000","last_peered":"2023-11-24T06:17:18.813754+0000","last_clean":"2023-11-24T06:17:18.813754+0000","last_became_active":"2023-11-24T06:17:17.413366+0000","last_became_peered":"2023-11-24T06:17:17.413366+0000","last_unstale":"2023-11-24T06:17:18.813754+0000","last_undegraded":"2023-11-24T06:17:18.813754+0000","last_fullsized":"2023-11-24T06:17:18.813754+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:15:54.423492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819472+0000","last_change":"2023-11-24T06:17:16.424845+0000","last_active":"2023-11-24T06:17:18.819472+0000","last_peered":"2023-11-24T06:17:18.819472+0000","last_clean":"2023-11-24T06:17:18.819472+0000","last_became_active":"2023-11-24T06:17:16.424563+0000","last_became_peered":"2023-11-24T06:17:16.424563+0000","last_unstale":"2023-11-24T06:17:18.819472+0000","last_undegraded":"2023-11-24T06:17:18.819472+0000","last_fullsized":"2023-11-24T06:17:18.819472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:07.073655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387595+0000","last_change":"2023-11-24T06:17:17.420812+0000","last_active":"2023-11-24T06:17:18.387595+0000","last_peered":"2023-11-24T06:17:18.387595+0000","last_clean":"2023-11-24T06:17:18.387595+0000","last_became_active":"2023-11-24T06:17:17.419365+0000","last_became_peered":"2023-11-24T06:17:17.419365+0000","last_unstale":"2023-11-24T06:17:18.387595+0000","last_undegraded":"2023-11-24T06:17:18.387595+0000","last_fullsized":"2023-11-24T06:17:18.387595+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:39:42.344668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391419+0000","last_change":"2023-11-24T06:17:16.424980+0000","last_active":"2023-11-24T06:17:18.391419+0000","last_peered":"2023-11-24T06:17:18.391419+0000","last_clean":"2023-11-24T06:17:18.391419+0000","last_became_active":"2023-11-24T06:17:16.424673+0000","last_became_peered":"2023-11-24T06:17:16.424673+0000","last_unstale":"2023-11-24T06:17:18.391419+0000","last_undegraded":"2023-11-24T06:17:18.391419+0000","last_fullsized":"2023-11-24T06:17:18.391419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:38.228122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.390926+0000","last_change":"2023-11-24T06:17:17.412862+0000","last_active":"2023-11-24T06:17:18.390926+0000","last_peered":"2023-11-24T06:17:18.390926+0000","last_clean":"2023-11-24T06:17:18.390926+0000","last_became_active":"2023-11-24T06:17:17.412573+0000","last_became_peered":"2023-11-24T06:17:17.412573+0000","last_unstale":"2023-11-24T06:17:18.390926+0000","last_undegraded":"2023-11-24T06:17:18.390926+0000","last_fullsized":"2023-11-24T06:17:18.390926+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:50:21.979650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.388048+0000","last_change":"2023-11-24T06:17:16.430350+0000","last_active":"2023-11-24T06:17:18.388048+0000","last_peered":"2023-11-24T06:17:18.388048+0000","last_clean":"2023-11-24T06:17:18.388048+0000","last_became_active":"2023-11-24T06:17:16.429825+0000","last_became_peered":"2023-11-24T06:17:16.429825+0000","last_unstale":"2023-11-24T06:17:18.388048+0000","last_undegraded":"2023-11-24T06:17:18.388048+0000","last_fullsized":"2023-11-24T06:17:18.388048+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:11.836881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.394025+0000","last_change":"2023-11-24T06:17:17.421289+0000","last_active":"2023-11-24T06:17:18.394025+0000","last_peered":"2023-11-24T06:17:18.394025+0000","last_clean":"2023-11-24T06:17:18.394025+0000","last_became_active":"2023-11-24T06:17:17.420149+0000","last_became_peered":"2023-11-24T06:17:17.420149+0000","last_unstale":"2023-11-24T06:17:18.394025+0000","last_undegraded":"2023-11-24T06:17:18.394025+0000","last_fullsized":"2023-11-24T06:17:18.394025+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:39:42.344668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819095+0000","last_change":"2023-11-24T06:17:16.423584+0000","last_active":"2023-11-24T06:17:18.819095+0000","last_peered":"2023-11-24T06:17:18.819095+0000","last_clean":"2023-11-24T06:17:18.819095+0000","last_became_active":"2023-11-24T06:17:16.423260+0000","last_became_peered":"2023-11-24T06:17:16.423260+0000","last_unstale":"2023-11-24T06:17:18.819095+0000","last_undegraded":"2023-11-24T06:17:18.819095+0000","last_fullsized":"2023-11-24T06:17:18.819095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:28:07.275105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":174,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:17:17.408126+0000","last_change":"2023-11-24T06:15:42.021608+0000","last_active":"2023-11-24T06:17:17.408126+0000","last_peered":"2023-11-24T06:17:17.408126+0000","last_clean":"2023-11-24T06:17:17.408126+0000","last_became_active":"2023-11-24T06:15:42.018847+0000","last_became_peered":"2023-11-24T06:15:42.018847+0000","last_unstale":"2023-11-24T06:17:17.408126+0000","last_undegraded":"2023-11-24T06:17:17.408126+0000","last_fullsized":"2023-11-24T06:17:17.408126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_clean_scrub_stamp":"2023-11-24T06:15:40.694368+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:41.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":987136,"data_stored":744396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.663},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.6359999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":2.1720000000000002},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.7719999999999998,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":1.3400000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":4.0579999999999998,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.778}]},{"osd":3,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":3.4239999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":1.585},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":3.0569999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":1.857}]},{"osd":4,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"min":{"1min":1.016,"5min":1.016,"15min":1.016},"max":{"1min":5.7560000000000002,"5min":5.7560000000000002,"15min":5.7560000000000002},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":2.681,"5min":2.681,"15min":2.681},"last":1.724}]},{"osd":5,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":2.117,"5min":2.117,"15min":2.117},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":6.6260000000000003,"5min":6.6260000000000003,"15min":6.6260000000000003},"last":2.0960000000000001},{"interface":"front","average":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"min":{"1min":1.21,"5min":1.21,"15min":1.21},"max":{"1min":6.4169999999999998,"5min":6.4169999999999998,"15min":6.4169999999999998},"last":1.484}]},{"osd":6,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"min":{"1min":1.306,"5min":1.306,"15min":1.306},"max":{"1min":4.4050000000000002,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":1.913},{"interface":"front","average":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"min":{"1min":1.337,"5min":1.337,"15min":1.337},"max":{"1min":3.8540000000000001,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":2.0139999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":397312,"data_stored":154042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.871},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.3740000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.2809999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.488}]},{"osd":3,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":1.556},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.593}]},{"osd":5,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.631,"5min":1.631,"15min":1.631},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":2.6989999999999998,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":2.4740000000000002,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.6339999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.702,"5min":1.702,"15min":1.702},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.766,"5min":1.766,"15min":1.766},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":1.7769999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.788,"5min":1.788,"15min":1.788},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":3.4550000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":1.708},{"interface":"front","average":{"1min":1.774,"5min":1.774,"15min":1.774},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":1.7350000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542166,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":148254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.1779999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.054}]},{"osd":2,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.343},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.401}]},{"osd":3,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.409,"15min":1.409},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.601,"15min":1.601},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.5249999999999999,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":1.6140000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7290000000000001,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.4670000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.5179999999999998,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.1180000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.788,"5min":1.788,"15min":1.788},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.6880000000000002,"5min":2.6880000000000002,"15min":2.6880000000000002},"last":1.819},{"interface":"front","average":{"1min":1.762,"5min":1.762,"15min":1.762},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.6699999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":148310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.657},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.94599999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.861},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.4390000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.24},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.7609999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.419},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.3860000000000001,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":1.379}]},{"osd":4,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.542},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":1.504}]},{"osd":6,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.6480000000000001,"5min":2.6480000000000001,"15min":2.6480000000000001},"last":1.294},{"interface":"front","average":{"1min":1.631,"5min":1.631,"15min":1.631},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":3.1560000000000001,"5min":3.1560000000000001,"15min":3.1560000000000001},"last":1.337}]},{"osd":7,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.659,"15min":1.659},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":3.2949999999999999,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.474},{"interface":"front","average":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":4.1779999999999999,"5min":4.1779999999999999,"15min":4.1779999999999999},"last":1.1679999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":405504,"data_stored":155572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.879},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.232}]},{"osd":1,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.0249999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.129}]},{"osd":4,"last update":"Fri Nov 24 06:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.1719999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.569,"5min":1.569,"15min":1.569},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":3.0089999999999999,"5min":3.0089999999999999,"15min":3.0089999999999999},"last":1.286}]},{"osd":6,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.377},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.427}]},{"osd":7,"last update":"Fri Nov 24 06:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.573,"15min":1.573},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":1.079,"5min":1.079,"15min":1.079},"max":{"1min":3.1280000000000001,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.5880000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":147708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.5390000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":0.57799999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":0.59599999999999997},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.96799999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.177}]},{"osd":6,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":3.0499999999999998,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.9049999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":0.95399999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":405504,"data_stored":155572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.92200000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.1910000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":1.228}]},{"osd":4,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.55}]},{"osd":5,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.153},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":2.355,"5min":2.355,"15min":2.355},"last":1.494}]},{"osd":6,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.385},{"interface":"front","average":{"1min":1.409,"5min":1.409,"15min":1.409},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.413}]},{"osd":7,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.637}]}]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":966656,"data_stored":737287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":0.69699999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":1.6699999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1.522}]},{"osd":4,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":1.405},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.347}]},{"osd":5,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.792},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.603}]},{"osd":6,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.3159999999999998,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.8540000000000001},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.5699999999999998,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.752}]},{"osd":7,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.988},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.925}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:17:22.252 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:17:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.513+0000 7f8233fff640 1 Processor -- start 2023-11-24T06:17:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.513+0000 7f8233fff640 1 -- start start 2023-11-24T06:17:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.515+0000 7f8233fff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8234162670 0x7f82340cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.515+0000 7f8233fff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82340cf9d0 con 0x7f82340cf590 2023-11-24T06:17:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.515+0000 7f8233fff640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8234165ca0 con 0x7f823414b1c0 2023-11-24T06:17:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.515+0000 7f8233fff640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8234165f80 con 0x7f8234162670 2023-11-24T06:17:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.515+0000 7f82337fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f82340cf590 0x7f823406c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50868/0 (socket says 172.21.15.5:50868) 2023-11-24T06:17:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.516+0000 7f82337fe640 1 -- 172.21.15.5:0/327667018 learned_addr learned my addr 172.21.15.5:0/327667018 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.516+0000 7f82327fc640 1 --2- 172.21.15.5:0/327667018 >> v2:172.21.15.6:3300/0 conn(0x7f8234162670 0x7f82340cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.516+0000 7f8231ffb640 1 -- 172.21.15.5:0/327667018 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355756891 0 0) 0x7f82340cf9d0 con 0x7f82340cf590 2023-11-24T06:17:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.516+0000 7f8231ffb640 1 -- 172.21.15.5:0/327667018 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8220003520 con 0x7f82340cf590 2023-11-24T06:17:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.516+0000 7f8231ffb640 1 -- 172.21.15.5:0/327667018 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 847225170 0 0) 0x7f8220003520 con 0x7f82340cf590 2023-11-24T06:17:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.516+0000 7f8231ffb640 1 -- 172.21.15.5:0/327667018 >> v2:172.21.15.6:3300/0 conn(0x7f8234162670 msgr2=0x7f82340cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.516+0000 7f8231ffb640 1 --2- 172.21.15.5:0/327667018 >> v2:172.21.15.6:3300/0 conn(0x7f8234162670 0x7f82340cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.516+0000 7f8231ffb640 1 -- 172.21.15.5:0/327667018 >> v1:172.21.15.6:6789/0 conn(0x7f823414b1c0 legacy=0x7f82341651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.516+0000 7f8231ffb640 1 -- 172.21.15.5:0/327667018 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8234161e70 con 0x7f82340cf590 2023-11-24T06:17:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.516+0000 7f8231ffb640 1 -- 172.21.15.5:0/327667018 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f822c001670 con 0x7f82340cf590 2023-11-24T06:17:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.517+0000 7f8231ffb640 1 -- 172.21.15.5:0/327667018 >> v1:172.21.15.5:6789/0 conn(0x7f82340cf590 legacy=0x7f823406c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.517+0000 7f8231ffb640 1 --2- 172.21.15.5:0/327667018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82200039e0 0x7f8220003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.517+0000 7f8231ffb640 1 --2- 172.21.15.5:0/327667018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82200047c0 0x7f8220004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.517+0000 7f8231ffb640 1 --2- 172.21.15.5:0/327667018 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82200051f0 0x7f822000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.517+0000 7f8231ffb640 1 -- 172.21.15.5:0/327667018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8234165f80 con 0x7f82200047c0 2023-11-24T06:17:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.517+0000 7f8231ffb640 1 -- 172.21.15.5:0/327667018 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822000c150 con 0x7f82200039e0 2023-11-24T06:17:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.517+0000 7f8231ffb640 1 -- 172.21.15.5:0/327667018 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822000c310 con 0x7f82200051f0 2023-11-24T06:17:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.517+0000 7f82327fc640 1 --2- 172.21.15.5:0/327667018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82200047c0 0x7f8220004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.517+0000 7f8232ffd640 1 --2- 172.21.15.5:0/327667018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82200039e0 0x7f8220003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.517+0000 7f82327fc640 1 -- 172.21.15.5:0/327667018 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82200051f0 msgr2=0x7f822000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.517+0000 7f82327fc640 1 --2- 172.21.15.5:0/327667018 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82200051f0 0x7f822000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.517+0000 7f82327fc640 1 -- 172.21.15.5:0/327667018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82200039e0 msgr2=0x7f8220003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.517+0000 7f82327fc640 1 --2- 172.21.15.5:0/327667018 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82200039e0 0x7f8220003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.517+0000 7f82327fc640 1 -- 172.21.15.5:0/327667018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8220003520 con 0x7f82200047c0 2023-11-24T06:17:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.518+0000 7f82337fe640 1 -- 172.21.15.5:0/327667018 reap_dead start 2023-11-24T06:17:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.519+0000 7f82327fc640 1 --2- 172.21.15.5:0/327667018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82200047c0 0x7f8220004c40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f821c009ac0 tx=0x7f82200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f6292164366f9a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.519+0000 7f8231ffb640 1 -- 172.21.15.5:0/327667018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c02c6b0 con 0x7f82200047c0 2023-11-24T06:17:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.519+0000 7f8231ffb640 1 -- 172.21.15.5:0/327667018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821c02c810 con 0x7f82200047c0 2023-11-24T06:17:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.519+0000 7f8231ffb640 1 -- 172.21.15.5:0/327667018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c02cba0 con 0x7f82200047c0 2023-11-24T06:17:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.519+0000 7f8233fff640 1 -- 172.21.15.5:0/327667018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82200047c0 msgr2=0x7f8220004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.519+0000 7f8233fff640 1 --2- 172.21.15.5:0/327667018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82200047c0 0x7f8220004c40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f821c009ac0 tx=0x7f82200057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.519+0000 7f8233fff640 1 -- 172.21.15.5:0/327667018 shutdown_connections 2023-11-24T06:17:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.519+0000 7f8233fff640 1 --2- 172.21.15.5:0/327667018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82200047c0 0x7f8220004c40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.519+0000 7f8233fff640 1 -- 172.21.15.5:0/327667018 >> 172.21.15.5:0/327667018 conn(0x7f823406a860 msgr2=0x7f823406ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.520+0000 7f8233fff640 1 -- 172.21.15.5:0/327667018 shutdown_connections 2023-11-24T06:17:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.520+0000 7f8233fff640 1 -- 172.21.15.5:0/327667018 wait complete. 2023-11-24T06:17:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.521+0000 7f8233fff640 1 Processor -- start 2023-11-24T06:17:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.522+0000 7f8233fff640 1 -- start start 2023-11-24T06:17:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.522+0000 7f8233fff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82200047c0 0x7f823413e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.522+0000 7f8233fff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82340afcf0 0x7f8234160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.522+0000 7f8233fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82340b0170 0x7f82341610f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.523+0000 7f8233fff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8234161630 con 0x7f82340b0170 2023-11-24T06:17:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.523+0000 7f8233fff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823413d8e0 con 0x7f82200047c0 2023-11-24T06:17:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.523+0000 7f8233fff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823413dbc0 con 0x7f82340afcf0 2023-11-24T06:17:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.523+0000 7f82337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82340b0170 0x7f82341610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.523+0000 7f82337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82340b0170 0x7f82341610f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50458/0 (socket says 172.21.15.5:50458) 2023-11-24T06:17:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.523+0000 7f82337fe640 1 -- 172.21.15.5:0/279390289 learned_addr learned my addr 172.21.15.5:0/279390289 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.523+0000 7f8232ffd640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82200047c0 0x7f823413e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.523+0000 7f82327fc640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82340afcf0 0x7f8234160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.523+0000 7f82337fe640 1 -- 172.21.15.5:0/279390289 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82340afcf0 msgr2=0x7f8234160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.523+0000 7f82337fe640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82340afcf0 0x7f8234160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.523+0000 7f82337fe640 1 -- 172.21.15.5:0/279390289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82200047c0 msgr2=0x7f823413e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.523+0000 7f82337fe640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82200047c0 0x7f823413e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.523+0000 7f82337fe640 1 -- 172.21.15.5:0/279390289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823413dea0 con 0x7f82340b0170 2023-11-24T06:17:22.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.523+0000 7f8232ffd640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82200047c0 0x7f823413e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:22.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.524+0000 7f82337fe640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82340b0170 0x7f82341610f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f822c000f60 tx=0x7f822c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:22.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.524+0000 7f822b7fe640 1 -- 172.21.15.5:0/279390289 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c007230 con 0x7f82340b0170 2023-11-24T06:17:22.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.524+0000 7f822b7fe640 1 -- 172.21.15.5:0/279390289 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c007390 con 0x7f82340b0170 2023-11-24T06:17:22.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.524+0000 7f822b7fe640 1 -- 172.21.15.5:0/279390289 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c00ce50 con 0x7f82340b0170 2023-11-24T06:17:22.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.524+0000 7f8233fff640 1 -- 172.21.15.5:0/279390289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823413e180 con 0x7f82340b0170 2023-11-24T06:17:22.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.524+0000 7f8233fff640 1 -- 172.21.15.5:0/279390289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823413cc40 con 0x7f82340b0170 2023-11-24T06:17:22.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.525+0000 7f8233fff640 1 -- 172.21.15.5:0/279390289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823413e6a0 con 0x7f82340b0170 2023-11-24T06:17:22.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.526+0000 7f822b7fe640 1 -- 172.21.15.5:0/279390289 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f822c0047e0 con 0x7f82340b0170 2023-11-24T06:17:22.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.527+0000 7f822b7fe640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f821406ae40 0x7f821406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:22.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.527+0000 7f822b7fe640 1 -- 172.21.15.5:0/279390289 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f822c088c30 con 0x7f82340b0170 2023-11-24T06:17:22.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.527+0000 7f8232ffd640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f821406ae40 0x7f821406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:22.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.528+0000 7f8232ffd640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f821406ae40 0x7f821406d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8224000f80 tx=0x7f8224009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:22.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.532+0000 7f822b7fe640 1 -- 172.21.15.5:0/279390289 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f823413e6a0 con 0x7f82340b0170 2023-11-24T06:17:22.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.716+0000 7f8233fff640 1 -- 172.21.15.5:0/279390289 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8234161b50 con 0x7f821406ae40 2023-11-24T06:17:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.724+0000 7f822b7fe640 1 -- 172.21.15.5:0/279390289 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287046 (secure 0 0 0) 0x7f822400d050 con 0x7f821406ae40 2023-11-24T06:17:22.729 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:22.733 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-11-24T06:17:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.731+0000 7f8233fff640 1 -- 172.21.15.5:0/279390289 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f821406ae40 msgr2=0x7f821406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.731+0000 7f8233fff640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f821406ae40 0x7f821406d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8224000f80 tx=0x7f8224009040 comp rx=0 tx=0).stop 2023-11-24T06:17:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.731+0000 7f8233fff640 1 -- 172.21.15.5:0/279390289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82340b0170 msgr2=0x7f82341610f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.731+0000 7f8233fff640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82340b0170 0x7f82341610f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f822c000f60 tx=0x7f822c00cb10 comp rx=0 tx=0).stop 2023-11-24T06:17:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.731+0000 7f8233fff640 1 -- 172.21.15.5:0/279390289 shutdown_connections 2023-11-24T06:17:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.731+0000 7f8233fff640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f821406ae40 0x7f821406d300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.731+0000 7f8233fff640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f82340afcf0 0x7f8234160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.731+0000 7f8233fff640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82340b0170 0x7f82341610f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.731+0000 7f8233fff640 1 --2- 172.21.15.5:0/279390289 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f82200047c0 0x7f823413e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.731+0000 7f8233fff640 1 -- 172.21.15.5:0/279390289 >> 172.21.15.5:0/279390289 conn(0x7f823406a860 msgr2=0x7f823406d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.731+0000 7f8233fff640 1 -- 172.21.15.5:0/279390289 shutdown_connections 2023-11-24T06:17:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:22.731+0000 7f8233fff640 1 -- 172.21.15.5:0/279390289 wait complete. 2023-11-24T06:17:22.753 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-11-24T06:17:22.115806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":116,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":241192,"kb_used_data":4212,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733528,"statfs":{"total":773094113280,"available":772847132672,"internally_reserved":0,"allocated":4313088,"data_stored":2391141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001825"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820444+0000","last_change":"2023-11-24T06:17:17.425737+0000","last_active":"2023-11-24T06:17:18.820444+0000","last_peered":"2023-11-24T06:17:18.820444+0000","last_clean":"2023-11-24T06:17:18.820444+0000","last_became_active":"2023-11-24T06:17:17.425257+0000","last_became_peered":"2023-11-24T06:17:17.425257+0000","last_unstale":"2023-11-24T06:17:18.820444+0000","last_undegraded":"2023-11-24T06:17:18.820444+0000","last_fullsized":"2023-11-24T06:17:18.820444+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:54:03.724108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.388245+0000","last_change":"2023-11-24T06:17:17.415520+0000","last_active":"2023-11-24T06:17:18.388245+0000","last_peered":"2023-11-24T06:17:18.388245+0000","last_clean":"2023-11-24T06:17:18.388245+0000","last_became_active":"2023-11-24T06:17:17.413377+0000","last_became_peered":"2023-11-24T06:17:17.413377+0000","last_unstale":"2023-11-24T06:17:18.388245+0000","last_undegraded":"2023-11-24T06:17:18.388245+0000","last_fullsized":"2023-11-24T06:17:18.388245+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:28.886659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.822352+0000","last_change":"2023-11-24T06:17:17.419104+0000","last_active":"2023-11-24T06:17:18.822352+0000","last_peered":"2023-11-24T06:17:18.822352+0000","last_clean":"2023-11-24T06:17:18.822352+0000","last_became_active":"2023-11-24T06:17:17.418752+0000","last_became_peered":"2023-11-24T06:17:17.418752+0000","last_unstale":"2023-11-24T06:17:18.822352+0000","last_undegraded":"2023-11-24T06:17:18.822352+0000","last_fullsized":"2023-11-24T06:17:18.822352+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:34:30.808706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819988+0000","last_change":"2023-11-24T06:17:17.413918+0000","last_active":"2023-11-24T06:17:18.819988+0000","last_peered":"2023-11-24T06:17:18.819988+0000","last_clean":"2023-11-24T06:17:18.819988+0000","last_became_active":"2023-11-24T06:17:17.413523+0000","last_became_peered":"2023-11-24T06:17:17.413523+0000","last_unstale":"2023-11-24T06:17:18.819988+0000","last_undegraded":"2023-11-24T06:17:18.819988+0000","last_fullsized":"2023-11-24T06:17:18.819988+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:12.852430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.816481+0000","last_change":"2023-11-24T06:17:17.423600+0000","last_active":"2023-11-24T06:17:18.816481+0000","last_peered":"2023-11-24T06:17:18.816481+0000","last_clean":"2023-11-24T06:17:18.816481+0000","last_became_active":"2023-11-24T06:17:17.421248+0000","last_became_peered":"2023-11-24T06:17:17.421248+0000","last_unstale":"2023-11-24T06:17:18.816481+0000","last_undegraded":"2023-11-24T06:17:18.816481+0000","last_fullsized":"2023-11-24T06:17:18.816481+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:56:00.416571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387593+0000","last_change":"2023-11-24T06:17:17.417869+0000","last_active":"2023-11-24T06:17:18.387593+0000","last_peered":"2023-11-24T06:17:18.387593+0000","last_clean":"2023-11-24T06:17:18.387593+0000","last_became_active":"2023-11-24T06:17:17.417580+0000","last_became_peered":"2023-11-24T06:17:17.417580+0000","last_unstale":"2023-11-24T06:17:18.387593+0000","last_undegraded":"2023-11-24T06:17:18.387593+0000","last_fullsized":"2023-11-24T06:17:18.387593+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:28.122398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392453+0000","last_change":"2023-11-24T06:17:17.420507+0000","last_active":"2023-11-24T06:17:18.392453+0000","last_peered":"2023-11-24T06:17:18.392453+0000","last_clean":"2023-11-24T06:17:18.392453+0000","last_became_active":"2023-11-24T06:17:17.420168+0000","last_became_peered":"2023-11-24T06:17:17.420168+0000","last_unstale":"2023-11-24T06:17:18.392453+0000","last_undegraded":"2023-11-24T06:17:18.392453+0000","last_fullsized":"2023-11-24T06:17:18.392453+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:36:54.531275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.390465+0000","last_change":"2023-11-24T06:17:17.422042+0000","last_active":"2023-11-24T06:17:18.390465+0000","last_peered":"2023-11-24T06:17:18.390465+0000","last_clean":"2023-11-24T06:17:18.390465+0000","last_became_active":"2023-11-24T06:17:17.421715+0000","last_became_peered":"2023-11-24T06:17:17.421715+0000","last_unstale":"2023-11-24T06:17:18.390465+0000","last_undegraded":"2023-11-24T06:17:18.390465+0000","last_fullsized":"2023-11-24T06:17:18.390465+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:53.702868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.393086+0000","last_change":"2023-11-24T06:17:17.412252+0000","last_active":"2023-11-24T06:17:18.393086+0000","last_peered":"2023-11-24T06:17:18.393086+0000","last_clean":"2023-11-24T06:17:18.393086+0000","last_became_active":"2023-11-24T06:17:17.411979+0000","last_became_peered":"2023-11-24T06:17:17.411979+0000","last_unstale":"2023-11-24T06:17:18.393086+0000","last_undegraded":"2023-11-24T06:17:18.393086+0000","last_fullsized":"2023-11-24T06:17:18.393086+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:53.702868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820869+0000","last_change":"2023-11-24T06:17:17.425430+0000","last_active":"2023-11-24T06:17:18.820869+0000","last_peered":"2023-11-24T06:17:18.820869+0000","last_clean":"2023-11-24T06:17:18.820869+0000","last_became_active":"2023-11-24T06:17:17.424546+0000","last_became_peered":"2023-11-24T06:17:17.424546+0000","last_unstale":"2023-11-24T06:17:18.820869+0000","last_undegraded":"2023-11-24T06:17:18.820869+0000","last_fullsized":"2023-11-24T06:17:18.820869+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:01.018319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.394026+0000","last_change":"2023-11-24T06:17:17.427370+0000","last_active":"2023-11-24T06:17:18.394026+0000","last_peered":"2023-11-24T06:17:18.394026+0000","last_clean":"2023-11-24T06:17:18.394026+0000","last_became_active":"2023-11-24T06:17:17.426906+0000","last_became_peered":"2023-11-24T06:17:17.426906+0000","last_unstale":"2023-11-24T06:17:18.394026+0000","last_undegraded":"2023-11-24T06:17:18.394026+0000","last_fullsized":"2023-11-24T06:17:18.394026+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:18.330889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.821818+0000","last_change":"2023-11-24T06:17:17.417524+0000","last_active":"2023-11-24T06:17:18.821818+0000","last_peered":"2023-11-24T06:17:18.821818+0000","last_clean":"2023-11-24T06:17:18.821818+0000","last_became_active":"2023-11-24T06:17:17.417254+0000","last_became_peered":"2023-11-24T06:17:17.417254+0000","last_unstale":"2023-11-24T06:17:18.821818+0000","last_undegraded":"2023-11-24T06:17:18.821818+0000","last_fullsized":"2023-11-24T06:17:18.821818+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:08:42.324430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.816669+0000","last_change":"2023-11-24T06:17:17.426552+0000","last_active":"2023-11-24T06:17:18.816669+0000","last_peered":"2023-11-24T06:17:18.816669+0000","last_clean":"2023-11-24T06:17:18.816669+0000","last_became_active":"2023-11-24T06:17:17.426239+0000","last_became_peered":"2023-11-24T06:17:17.426239+0000","last_unstale":"2023-11-24T06:17:18.816669+0000","last_undegraded":"2023-11-24T06:17:18.816669+0000","last_fullsized":"2023-11-24T06:17:18.816669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:14:16.766695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387526+0000","last_change":"2023-11-24T06:17:17.425345+0000","last_active":"2023-11-24T06:17:18.387526+0000","last_peered":"2023-11-24T06:17:18.387526+0000","last_clean":"2023-11-24T06:17:18.387526+0000","last_became_active":"2023-11-24T06:17:17.425041+0000","last_became_peered":"2023-11-24T06:17:17.425041+0000","last_unstale":"2023-11-24T06:17:18.387526+0000","last_undegraded":"2023-11-24T06:17:18.387526+0000","last_fullsized":"2023-11-24T06:17:18.387526+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:04.241501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819585+0000","last_change":"2023-11-24T06:17:17.415267+0000","last_active":"2023-11-24T06:17:18.819585+0000","last_peered":"2023-11-24T06:17:18.819585+0000","last_clean":"2023-11-24T06:17:18.819585+0000","last_became_active":"2023-11-24T06:17:17.414973+0000","last_became_peered":"2023-11-24T06:17:17.414973+0000","last_unstale":"2023-11-24T06:17:18.819585+0000","last_undegraded":"2023-11-24T06:17:18.819585+0000","last_fullsized":"2023-11-24T06:17:18.819585+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:07.850355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391529+0000","last_change":"2023-11-24T06:17:17.419442+0000","last_active":"2023-11-24T06:17:18.391529+0000","last_peered":"2023-11-24T06:17:18.391529+0000","last_clean":"2023-11-24T06:17:18.391529+0000","last_became_active":"2023-11-24T06:17:17.419137+0000","last_became_peered":"2023-11-24T06:17:17.419137+0000","last_unstale":"2023-11-24T06:17:18.391529+0000","last_undegraded":"2023-11-24T06:17:18.391529+0000","last_fullsized":"2023-11-24T06:17:18.391529+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:01.018319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.394121+0000","last_change":"2023-11-24T06:17:17.425692+0000","last_active":"2023-11-24T06:17:18.394121+0000","last_peered":"2023-11-24T06:17:18.394121+0000","last_clean":"2023-11-24T06:17:18.394121+0000","last_became_active":"2023-11-24T06:17:17.422123+0000","last_became_peered":"2023-11-24T06:17:17.422123+0000","last_unstale":"2023-11-24T06:17:18.394121+0000","last_undegraded":"2023-11-24T06:17:18.394121+0000","last_fullsized":"2023-11-24T06:17:18.394121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:00.817489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.388264+0000","last_change":"2023-11-24T06:17:17.414817+0000","last_active":"2023-11-24T06:17:18.388264+0000","last_peered":"2023-11-24T06:17:18.388264+0000","last_clean":"2023-11-24T06:17:18.388264+0000","last_became_active":"2023-11-24T06:17:17.410953+0000","last_became_peered":"2023-11-24T06:17:17.410953+0000","last_unstale":"2023-11-24T06:17:18.388264+0000","last_undegraded":"2023-11-24T06:17:18.388264+0000","last_fullsized":"2023-11-24T06:17:18.388264+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:45.003151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387679+0000","last_change":"2023-11-24T06:17:17.421106+0000","last_active":"2023-11-24T06:17:18.387679+0000","last_peered":"2023-11-24T06:17:18.387679+0000","last_clean":"2023-11-24T06:17:18.387679+0000","last_became_active":"2023-11-24T06:17:17.420722+0000","last_became_peered":"2023-11-24T06:17:17.420722+0000","last_unstale":"2023-11-24T06:17:18.387679+0000","last_undegraded":"2023-11-24T06:17:18.387679+0000","last_fullsized":"2023-11-24T06:17:18.387679+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:18.330889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.817173+0000","last_change":"2023-11-24T06:17:17.422038+0000","last_active":"2023-11-24T06:17:18.817173+0000","last_peered":"2023-11-24T06:17:18.817173+0000","last_clean":"2023-11-24T06:17:18.817173+0000","last_became_active":"2023-11-24T06:17:17.421751+0000","last_became_peered":"2023-11-24T06:17:17.421751+0000","last_unstale":"2023-11-24T06:17:18.817173+0000","last_undegraded":"2023-11-24T06:17:18.817173+0000","last_fullsized":"2023-11-24T06:17:18.817173+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:04:51.481600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.393087+0000","last_change":"2023-11-24T06:17:17.411841+0000","last_active":"2023-11-24T06:17:18.393087+0000","last_peered":"2023-11-24T06:17:18.393087+0000","last_clean":"2023-11-24T06:17:18.393087+0000","last_became_active":"2023-11-24T06:17:17.410346+0000","last_became_peered":"2023-11-24T06:17:17.410346+0000","last_unstale":"2023-11-24T06:17:18.393087+0000","last_undegraded":"2023-11-24T06:17:18.393087+0000","last_fullsized":"2023-11-24T06:17:18.393087+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:58.239341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391983+0000","last_change":"2023-11-24T06:17:17.409418+0000","last_active":"2023-11-24T06:17:18.391983+0000","last_peered":"2023-11-24T06:17:18.391983+0000","last_clean":"2023-11-24T06:17:18.391983+0000","last_became_active":"2023-11-24T06:17:17.408278+0000","last_became_peered":"2023-11-24T06:17:17.408278+0000","last_unstale":"2023-11-24T06:17:18.391983+0000","last_undegraded":"2023-11-24T06:17:18.391983+0000","last_fullsized":"2023-11-24T06:17:18.391983+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:56.627061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.812709+0000","last_change":"2023-11-24T06:17:17.419339+0000","last_active":"2023-11-24T06:17:18.812709+0000","last_peered":"2023-11-24T06:17:18.812709+0000","last_clean":"2023-11-24T06:17:18.812709+0000","last_became_active":"2023-11-24T06:17:17.418936+0000","last_became_peered":"2023-11-24T06:17:17.418936+0000","last_unstale":"2023-11-24T06:17:18.812709+0000","last_undegraded":"2023-11-24T06:17:18.812709+0000","last_fullsized":"2023-11-24T06:17:18.812709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:25:15.440363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.818943+0000","last_change":"2023-11-24T06:17:17.415059+0000","last_active":"2023-11-24T06:17:18.818943+0000","last_peered":"2023-11-24T06:17:18.818943+0000","last_clean":"2023-11-24T06:17:18.818943+0000","last_became_active":"2023-11-24T06:17:17.414713+0000","last_became_peered":"2023-11-24T06:17:17.414713+0000","last_unstale":"2023-11-24T06:17:18.818943+0000","last_undegraded":"2023-11-24T06:17:18.818943+0000","last_fullsized":"2023-11-24T06:17:18.818943+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:28:48.589643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820521+0000","last_change":"2023-11-24T06:17:17.420165+0000","last_active":"2023-11-24T06:17:18.820521+0000","last_peered":"2023-11-24T06:17:18.820521+0000","last_clean":"2023-11-24T06:17:18.820521+0000","last_became_active":"2023-11-24T06:17:17.419860+0000","last_became_peered":"2023-11-24T06:17:17.419860+0000","last_unstale":"2023-11-24T06:17:18.820521+0000","last_undegraded":"2023-11-24T06:17:18.820521+0000","last_fullsized":"2023-11-24T06:17:18.820521+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:23:58.898486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.388372+0000","last_change":"2023-11-24T06:17:17.422622+0000","last_active":"2023-11-24T06:17:18.388372+0000","last_peered":"2023-11-24T06:17:18.388372+0000","last_clean":"2023-11-24T06:17:18.388372+0000","last_became_active":"2023-11-24T06:17:17.420794+0000","last_became_peered":"2023-11-24T06:17:17.420794+0000","last_unstale":"2023-11-24T06:17:18.388372+0000","last_undegraded":"2023-11-24T06:17:18.388372+0000","last_fullsized":"2023-11-24T06:17:18.388372+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:04.564962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820115+0000","last_change":"2023-11-24T06:17:17.417600+0000","last_active":"2023-11-24T06:17:18.820115+0000","last_peered":"2023-11-24T06:17:18.820115+0000","last_clean":"2023-11-24T06:17:18.820115+0000","last_became_active":"2023-11-24T06:17:17.413146+0000","last_became_peered":"2023-11-24T06:17:17.413146+0000","last_unstale":"2023-11-24T06:17:18.820115+0000","last_undegraded":"2023-11-24T06:17:18.820115+0000","last_fullsized":"2023-11-24T06:17:18.820115+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:58.239341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.821699+0000","last_change":"2023-11-24T06:17:17.409350+0000","last_active":"2023-11-24T06:17:18.821699+0000","last_peered":"2023-11-24T06:17:18.821699+0000","last_clean":"2023-11-24T06:17:18.821699+0000","last_became_active":"2023-11-24T06:17:17.409064+0000","last_became_peered":"2023-11-24T06:17:17.409064+0000","last_unstale":"2023-11-24T06:17:18.821699+0000","last_undegraded":"2023-11-24T06:17:18.821699+0000","last_fullsized":"2023-11-24T06:17:18.821699+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:48.982744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392233+0000","last_change":"2023-11-24T06:17:17.419291+0000","last_active":"2023-11-24T06:17:18.392233+0000","last_peered":"2023-11-24T06:17:18.392233+0000","last_clean":"2023-11-24T06:17:18.392233+0000","last_became_active":"2023-11-24T06:17:17.418942+0000","last_became_peered":"2023-11-24T06:17:17.418942+0000","last_unstale":"2023-11-24T06:17:18.392233+0000","last_undegraded":"2023-11-24T06:17:18.392233+0000","last_fullsized":"2023-11-24T06:17:18.392233+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:12.852430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.393183+0000","last_change":"2023-11-24T06:17:17.418613+0000","last_active":"2023-11-24T06:17:18.393183+0000","last_peered":"2023-11-24T06:17:18.393183+0000","last_clean":"2023-11-24T06:17:18.393183+0000","last_became_active":"2023-11-24T06:17:17.417943+0000","last_became_peered":"2023-11-24T06:17:17.417943+0000","last_unstale":"2023-11-24T06:17:18.393183+0000","last_undegraded":"2023-11-24T06:17:18.393183+0000","last_fullsized":"2023-11-24T06:17:18.393183+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:04.241501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819895+0000","last_change":"2023-11-24T06:17:17.425415+0000","last_active":"2023-11-24T06:17:18.819895+0000","last_peered":"2023-11-24T06:17:18.819895+0000","last_clean":"2023-11-24T06:17:18.819895+0000","last_became_active":"2023-11-24T06:17:17.424917+0000","last_became_peered":"2023-11-24T06:17:17.424917+0000","last_unstale":"2023-11-24T06:17:18.819895+0000","last_undegraded":"2023-11-24T06:17:18.819895+0000","last_fullsized":"2023-11-24T06:17:18.819895+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:14.150534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.390466+0000","last_change":"2023-11-24T06:17:17.418861+0000","last_active":"2023-11-24T06:17:18.390466+0000","last_peered":"2023-11-24T06:17:18.390466+0000","last_clean":"2023-11-24T06:17:18.390466+0000","last_became_active":"2023-11-24T06:17:17.418279+0000","last_became_peered":"2023-11-24T06:17:17.418279+0000","last_unstale":"2023-11-24T06:17:18.390466+0000","last_undegraded":"2023-11-24T06:17:18.390466+0000","last_fullsized":"2023-11-24T06:17:18.390466+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:58.239341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.814567+0000","last_change":"2023-11-24T06:17:17.419892+0000","last_active":"2023-11-24T06:17:18.814567+0000","last_peered":"2023-11-24T06:17:18.814567+0000","last_clean":"2023-11-24T06:17:18.814567+0000","last_became_active":"2023-11-24T06:17:17.419612+0000","last_became_peered":"2023-11-24T06:17:17.419612+0000","last_unstale":"2023-11-24T06:17:18.814567+0000","last_undegraded":"2023-11-24T06:17:18.814567+0000","last_fullsized":"2023-11-24T06:17:18.814567+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:33.824783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820695+0000","last_change":"2023-11-24T06:17:16.425069+0000","last_active":"2023-11-24T06:17:18.820695+0000","last_peered":"2023-11-24T06:17:18.820695+0000","last_clean":"2023-11-24T06:17:18.820695+0000","last_became_active":"2023-11-24T06:17:16.424500+0000","last_became_peered":"2023-11-24T06:17:16.424500+0000","last_unstale":"2023-11-24T06:17:18.820695+0000","last_undegraded":"2023-11-24T06:17:18.820695+0000","last_fullsized":"2023-11-24T06:17:18.820695+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:27:29.090827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820809+0000","last_change":"2023-11-24T06:17:17.414213+0000","last_active":"2023-11-24T06:17:18.820809+0000","last_peered":"2023-11-24T06:17:18.820809+0000","last_clean":"2023-11-24T06:17:18.820809+0000","last_became_active":"2023-11-24T06:17:17.412662+0000","last_became_peered":"2023-11-24T06:17:17.412662+0000","last_unstale":"2023-11-24T06:17:18.820809+0000","last_undegraded":"2023-11-24T06:17:18.820809+0000","last_fullsized":"2023-11-24T06:17:18.820809+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:25:15.440363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.814815+0000","last_change":"2023-11-24T06:17:16.430731+0000","last_active":"2023-11-24T06:17:18.814815+0000","last_peered":"2023-11-24T06:17:18.814815+0000","last_clean":"2023-11-24T06:17:18.814815+0000","last_became_active":"2023-11-24T06:17:16.430185+0000","last_became_peered":"2023-11-24T06:17:16.430185+0000","last_unstale":"2023-11-24T06:17:18.814815+0000","last_undegraded":"2023-11-24T06:17:18.814815+0000","last_fullsized":"2023-11-24T06:17:18.814815+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:43:43.843622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392160+0000","last_change":"2023-11-24T06:17:17.416794+0000","last_active":"2023-11-24T06:17:18.392160+0000","last_peered":"2023-11-24T06:17:18.392160+0000","last_clean":"2023-11-24T06:17:18.392160+0000","last_became_active":"2023-11-24T06:17:17.416449+0000","last_became_peered":"2023-11-24T06:17:17.416449+0000","last_unstale":"2023-11-24T06:17:18.392160+0000","last_undegraded":"2023-11-24T06:17:18.392160+0000","last_fullsized":"2023-11-24T06:17:18.392160+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:24:04.806063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387692+0000","last_change":"2023-11-24T06:17:16.416080+0000","last_active":"2023-11-24T06:17:18.387692+0000","last_peered":"2023-11-24T06:17:18.387692+0000","last_clean":"2023-11-24T06:17:18.387692+0000","last_became_active":"2023-11-24T06:17:16.415751+0000","last_became_peered":"2023-11-24T06:17:16.415751+0000","last_unstale":"2023-11-24T06:17:18.387692+0000","last_undegraded":"2023-11-24T06:17:18.387692+0000","last_fullsized":"2023-11-24T06:17:18.387692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:00.002770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820309+0000","last_change":"2023-11-24T06:17:17.415686+0000","last_active":"2023-11-24T06:17:18.820309+0000","last_peered":"2023-11-24T06:17:18.820309+0000","last_clean":"2023-11-24T06:17:18.820309+0000","last_became_active":"2023-11-24T06:17:17.413606+0000","last_became_peered":"2023-11-24T06:17:17.413606+0000","last_unstale":"2023-11-24T06:17:18.820309+0000","last_undegraded":"2023-11-24T06:17:18.820309+0000","last_fullsized":"2023-11-24T06:17:18.820309+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:53.702868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.817166+0000","last_change":"2023-11-24T06:17:16.427095+0000","last_active":"2023-11-24T06:17:18.817166+0000","last_peered":"2023-11-24T06:17:18.817166+0000","last_clean":"2023-11-24T06:17:18.817166+0000","last_became_active":"2023-11-24T06:17:16.425462+0000","last_became_peered":"2023-11-24T06:17:16.425462+0000","last_unstale":"2023-11-24T06:17:18.817166+0000","last_undegraded":"2023-11-24T06:17:18.817166+0000","last_fullsized":"2023-11-24T06:17:18.817166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:23:42.085007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.393288+0000","last_change":"2023-11-24T06:17:17.418512+0000","last_active":"2023-11-24T06:17:18.393288+0000","last_peered":"2023-11-24T06:17:18.393288+0000","last_clean":"2023-11-24T06:17:18.393288+0000","last_became_active":"2023-11-24T06:17:17.418120+0000","last_became_peered":"2023-11-24T06:17:17.418120+0000","last_unstale":"2023-11-24T06:17:18.393288+0000","last_undegraded":"2023-11-24T06:17:18.393288+0000","last_fullsized":"2023-11-24T06:17:18.393288+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:28.122398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392516+0000","last_change":"2023-11-24T06:17:16.425516+0000","last_active":"2023-11-24T06:17:18.392516+0000","last_peered":"2023-11-24T06:17:18.392516+0000","last_clean":"2023-11-24T06:17:18.392516+0000","last_became_active":"2023-11-24T06:17:16.424425+0000","last_became_peered":"2023-11-24T06:17:16.424425+0000","last_unstale":"2023-11-24T06:17:18.392516+0000","last_undegraded":"2023-11-24T06:17:18.392516+0000","last_fullsized":"2023-11-24T06:17:18.392516+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:33:29.653709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.821224+0000","last_change":"2023-11-24T06:17:17.409205+0000","last_active":"2023-11-24T06:17:18.821224+0000","last_peered":"2023-11-24T06:17:18.821224+0000","last_clean":"2023-11-24T06:17:18.821224+0000","last_became_active":"2023-11-24T06:17:17.408937+0000","last_became_peered":"2023-11-24T06:17:17.408937+0000","last_unstale":"2023-11-24T06:17:18.821224+0000","last_undegraded":"2023-11-24T06:17:18.821224+0000","last_fullsized":"2023-11-24T06:17:18.821224+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:45:54.332599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391852+0000","last_change":"2023-11-24T06:17:16.424793+0000","last_active":"2023-11-24T06:17:18.391852+0000","last_peered":"2023-11-24T06:17:18.391852+0000","last_clean":"2023-11-24T06:17:18.391852+0000","last_became_active":"2023-11-24T06:17:16.423487+0000","last_became_peered":"2023-11-24T06:17:16.423487+0000","last_unstale":"2023-11-24T06:17:18.391852+0000","last_undegraded":"2023-11-24T06:17:18.391852+0000","last_fullsized":"2023-11-24T06:17:18.391852+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:53.398555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391212+0000","last_change":"2023-11-24T06:17:17.410932+0000","last_active":"2023-11-24T06:17:18.391212+0000","last_peered":"2023-11-24T06:17:18.391212+0000","last_clean":"2023-11-24T06:17:18.391212+0000","last_became_active":"2023-11-24T06:17:17.410624+0000","last_became_peered":"2023-11-24T06:17:17.410624+0000","last_unstale":"2023-11-24T06:17:18.391212+0000","last_undegraded":"2023-11-24T06:17:18.391212+0000","last_fullsized":"2023-11-24T06:17:18.391212+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:37:15.493664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387286+0000","last_change":"2023-11-24T06:17:16.414388+0000","last_active":"2023-11-24T06:17:18.387286+0000","last_peered":"2023-11-24T06:17:18.387286+0000","last_clean":"2023-11-24T06:17:18.387286+0000","last_became_active":"2023-11-24T06:17:16.414096+0000","last_became_peered":"2023-11-24T06:17:16.414096+0000","last_unstale":"2023-11-24T06:17:18.387286+0000","last_undegraded":"2023-11-24T06:17:18.387286+0000","last_fullsized":"2023-11-24T06:17:18.387286+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:52.687319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819361+0000","last_change":"2023-11-24T06:17:17.424413+0000","last_active":"2023-11-24T06:17:18.819361+0000","last_peered":"2023-11-24T06:17:18.819361+0000","last_clean":"2023-11-24T06:17:18.819361+0000","last_became_active":"2023-11-24T06:17:17.424105+0000","last_became_peered":"2023-11-24T06:17:17.424105+0000","last_unstale":"2023-11-24T06:17:18.819361+0000","last_undegraded":"2023-11-24T06:17:18.819361+0000","last_fullsized":"2023-11-24T06:17:18.819361+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:56.627061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391140+0000","last_change":"2023-11-24T06:17:16.425492+0000","last_active":"2023-11-24T06:17:18.391140+0000","last_peered":"2023-11-24T06:17:18.391140+0000","last_clean":"2023-11-24T06:17:18.391140+0000","last_became_active":"2023-11-24T06:17:16.424454+0000","last_became_peered":"2023-11-24T06:17:16.424454+0000","last_unstale":"2023-11-24T06:17:18.391140+0000","last_undegraded":"2023-11-24T06:17:18.391140+0000","last_fullsized":"2023-11-24T06:17:18.391140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:26:39.607586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.821910+0000","last_change":"2023-11-24T06:17:17.420252+0000","last_active":"2023-11-24T06:17:18.821910+0000","last_peered":"2023-11-24T06:17:18.821910+0000","last_clean":"2023-11-24T06:17:18.821910+0000","last_became_active":"2023-11-24T06:17:17.419989+0000","last_became_peered":"2023-11-24T06:17:17.419989+0000","last_unstale":"2023-11-24T06:17:18.821910+0000","last_undegraded":"2023-11-24T06:17:18.821910+0000","last_fullsized":"2023-11-24T06:17:18.821910+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:13:24.090710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387881+0000","last_change":"2023-11-24T06:17:16.429657+0000","last_active":"2023-11-24T06:17:18.387881+0000","last_peered":"2023-11-24T06:17:18.387881+0000","last_clean":"2023-11-24T06:17:18.387881+0000","last_became_active":"2023-11-24T06:17:16.429400+0000","last_became_peered":"2023-11-24T06:17:16.429400+0000","last_unstale":"2023-11-24T06:17:18.387881+0000","last_undegraded":"2023-11-24T06:17:18.387881+0000","last_fullsized":"2023-11-24T06:17:18.387881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:52.900957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387948+0000","last_change":"2023-11-24T06:17:17.416803+0000","last_active":"2023-11-24T06:17:18.387948+0000","last_peered":"2023-11-24T06:17:18.387948+0000","last_clean":"2023-11-24T06:17:18.387948+0000","last_became_active":"2023-11-24T06:17:17.416429+0000","last_became_peered":"2023-11-24T06:17:17.416429+0000","last_unstale":"2023-11-24T06:17:18.387948+0000","last_undegraded":"2023-11-24T06:17:18.387948+0000","last_fullsized":"2023-11-24T06:17:18.387948+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:27:55.321099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.814778+0000","last_change":"2023-11-24T06:17:16.409047+0000","last_active":"2023-11-24T06:17:18.814778+0000","last_peered":"2023-11-24T06:17:18.814778+0000","last_clean":"2023-11-24T06:17:18.814778+0000","last_became_active":"2023-11-24T06:17:16.408751+0000","last_became_peered":"2023-11-24T06:17:16.408751+0000","last_unstale":"2023-11-24T06:17:18.814778+0000","last_undegraded":"2023-11-24T06:17:18.814778+0000","last_fullsized":"2023-11-24T06:17:18.814778+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:13.134985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.388177+0000","last_change":"2023-11-24T06:17:17.422892+0000","last_active":"2023-11-24T06:17:18.388177+0000","last_peered":"2023-11-24T06:17:18.388177+0000","last_clean":"2023-11-24T06:17:18.388177+0000","last_became_active":"2023-11-24T06:17:17.422339+0000","last_became_peered":"2023-11-24T06:17:17.422339+0000","last_unstale":"2023-11-24T06:17:18.388177+0000","last_undegraded":"2023-11-24T06:17:18.388177+0000","last_fullsized":"2023-11-24T06:17:18.388177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:39.811108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.822149+0000","last_change":"2023-11-24T06:17:16.431179+0000","last_active":"2023-11-24T06:17:18.822149+0000","last_peered":"2023-11-24T06:17:18.822149+0000","last_clean":"2023-11-24T06:17:18.822149+0000","last_became_active":"2023-11-24T06:17:16.430031+0000","last_became_peered":"2023-11-24T06:17:16.430031+0000","last_unstale":"2023-11-24T06:17:18.822149+0000","last_undegraded":"2023-11-24T06:17:18.822149+0000","last_fullsized":"2023-11-24T06:17:18.822149+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:00.578190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.821841+0000","last_change":"2023-11-24T06:17:17.422022+0000","last_active":"2023-11-24T06:17:18.821841+0000","last_peered":"2023-11-24T06:17:18.821841+0000","last_clean":"2023-11-24T06:17:18.821841+0000","last_became_active":"2023-11-24T06:17:17.421753+0000","last_became_peered":"2023-11-24T06:17:17.421753+0000","last_unstale":"2023-11-24T06:17:18.821841+0000","last_undegraded":"2023-11-24T06:17:18.821841+0000","last_fullsized":"2023-11-24T06:17:18.821841+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:24:28.186658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.388092+0000","last_change":"2023-11-24T06:17:16.428348+0000","last_active":"2023-11-24T06:17:18.388092+0000","last_peered":"2023-11-24T06:17:18.388092+0000","last_clean":"2023-11-24T06:17:18.388092+0000","last_became_active":"2023-11-24T06:17:16.426800+0000","last_became_peered":"2023-11-24T06:17:16.426800+0000","last_unstale":"2023-11-24T06:17:18.388092+0000","last_undegraded":"2023-11-24T06:17:18.388092+0000","last_fullsized":"2023-11-24T06:17:18.388092+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:23:57.882937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819403+0000","last_change":"2023-11-24T06:17:17.423622+0000","last_active":"2023-11-24T06:17:18.819403+0000","last_peered":"2023-11-24T06:17:18.819403+0000","last_clean":"2023-11-24T06:17:18.819403+0000","last_became_active":"2023-11-24T06:17:17.421139+0000","last_became_peered":"2023-11-24T06:17:17.421139+0000","last_unstale":"2023-11-24T06:17:18.819403+0000","last_undegraded":"2023-11-24T06:17:18.819403+0000","last_fullsized":"2023-11-24T06:17:18.819403+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:24:04.806063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392777+0000","last_change":"2023-11-24T06:17:16.419010+0000","last_active":"2023-11-24T06:17:18.392777+0000","last_peered":"2023-11-24T06:17:18.392777+0000","last_clean":"2023-11-24T06:17:18.392777+0000","last_became_active":"2023-11-24T06:17:16.418637+0000","last_became_peered":"2023-11-24T06:17:16.418637+0000","last_unstale":"2023-11-24T06:17:18.392777+0000","last_undegraded":"2023-11-24T06:17:18.392777+0000","last_fullsized":"2023-11-24T06:17:18.392777+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:34.496015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387745+0000","last_change":"2023-11-24T06:17:17.416817+0000","last_active":"2023-11-24T06:17:18.387745+0000","last_peered":"2023-11-24T06:17:18.387745+0000","last_clean":"2023-11-24T06:17:18.387745+0000","last_became_active":"2023-11-24T06:17:17.415352+0000","last_became_peered":"2023-11-24T06:17:17.415352+0000","last_unstale":"2023-11-24T06:17:18.387745+0000","last_undegraded":"2023-11-24T06:17:18.387745+0000","last_fullsized":"2023-11-24T06:17:18.387745+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:00.817489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819350+0000","last_change":"2023-11-24T06:17:16.423325+0000","last_active":"2023-11-24T06:17:18.819350+0000","last_peered":"2023-11-24T06:17:18.819350+0000","last_clean":"2023-11-24T06:17:18.819350+0000","last_became_active":"2023-11-24T06:17:16.423000+0000","last_became_peered":"2023-11-24T06:17:16.423000+0000","last_unstale":"2023-11-24T06:17:18.819350+0000","last_undegraded":"2023-11-24T06:17:18.819350+0000","last_fullsized":"2023-11-24T06:17:18.819350+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:42:34.496015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.390839+0000","last_change":"2023-11-24T06:17:17.418624+0000","last_active":"2023-11-24T06:17:18.390839+0000","last_peered":"2023-11-24T06:17:18.390839+0000","last_clean":"2023-11-24T06:17:18.390839+0000","last_became_active":"2023-11-24T06:17:17.418236+0000","last_became_peered":"2023-11-24T06:17:17.418236+0000","last_unstale":"2023-11-24T06:17:18.390839+0000","last_undegraded":"2023-11-24T06:17:18.390839+0000","last_fullsized":"2023-11-24T06:17:18.390839+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:23:58.898486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387160+0000","last_change":"2023-11-24T06:17:16.429252+0000","last_active":"2023-11-24T06:17:18.387160+0000","last_peered":"2023-11-24T06:17:18.387160+0000","last_clean":"2023-11-24T06:17:18.387160+0000","last_became_active":"2023-11-24T06:17:16.428863+0000","last_became_peered":"2023-11-24T06:17:16.428863+0000","last_unstale":"2023-11-24T06:17:18.387160+0000","last_undegraded":"2023-11-24T06:17:18.387160+0000","last_fullsized":"2023-11-24T06:17:18.387160+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:04:50.451978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820255+0000","last_change":"2023-11-24T06:17:17.415156+0000","last_active":"2023-11-24T06:17:18.820255+0000","last_peered":"2023-11-24T06:17:18.820255+0000","last_clean":"2023-11-24T06:17:18.820255+0000","last_became_active":"2023-11-24T06:17:17.414507+0000","last_became_peered":"2023-11-24T06:17:17.414507+0000","last_unstale":"2023-11-24T06:17:18.820255+0000","last_undegraded":"2023-11-24T06:17:18.820255+0000","last_fullsized":"2023-11-24T06:17:18.820255+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:53:16.931385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387783+0000","last_change":"2023-11-24T06:17:16.420651+0000","last_active":"2023-11-24T06:17:18.387783+0000","last_peered":"2023-11-24T06:17:18.387783+0000","last_clean":"2023-11-24T06:17:18.387783+0000","last_became_active":"2023-11-24T06:17:16.420041+0000","last_became_peered":"2023-11-24T06:17:16.420041+0000","last_unstale":"2023-11-24T06:17:18.387783+0000","last_undegraded":"2023-11-24T06:17:18.387783+0000","last_fullsized":"2023-11-24T06:17:18.387783+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:57.223792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392669+0000","last_change":"2023-11-24T06:17:17.417057+0000","last_active":"2023-11-24T06:17:18.392669+0000","last_peered":"2023-11-24T06:17:18.392669+0000","last_clean":"2023-11-24T06:17:18.392669+0000","last_became_active":"2023-11-24T06:17:17.416064+0000","last_became_peered":"2023-11-24T06:17:17.416064+0000","last_unstale":"2023-11-24T06:17:18.392669+0000","last_undegraded":"2023-11-24T06:17:18.392669+0000","last_fullsized":"2023-11-24T06:17:18.392669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:04:51.467527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820659+0000","last_change":"2023-11-24T06:17:16.420014+0000","last_active":"2023-11-24T06:17:18.820659+0000","last_peered":"2023-11-24T06:17:18.820659+0000","last_clean":"2023-11-24T06:17:18.820659+0000","last_became_active":"2023-11-24T06:17:16.419682+0000","last_became_peered":"2023-11-24T06:17:16.419682+0000","last_unstale":"2023-11-24T06:17:18.820659+0000","last_undegraded":"2023-11-24T06:17:18.820659+0000","last_fullsized":"2023-11-24T06:17:18.820659+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:50:20.964101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.822727+0000","last_change":"2023-11-24T06:17:17.413988+0000","last_active":"2023-11-24T06:17:18.822727+0000","last_peered":"2023-11-24T06:17:18.822727+0000","last_clean":"2023-11-24T06:17:18.822727+0000","last_became_active":"2023-11-24T06:17:17.413634+0000","last_became_peered":"2023-11-24T06:17:17.413634+0000","last_unstale":"2023-11-24T06:17:18.822727+0000","last_undegraded":"2023-11-24T06:17:18.822727+0000","last_fullsized":"2023-11-24T06:17:18.822727+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:31.519613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392673+0000","last_change":"2023-11-24T06:17:16.418846+0000","last_active":"2023-11-24T06:17:18.392673+0000","last_peered":"2023-11-24T06:17:18.392673+0000","last_clean":"2023-11-24T06:17:18.392673+0000","last_became_active":"2023-11-24T06:17:16.418582+0000","last_became_peered":"2023-11-24T06:17:16.418582+0000","last_unstale":"2023-11-24T06:17:18.392673+0000","last_undegraded":"2023-11-24T06:17:18.392673+0000","last_fullsized":"2023-11-24T06:17:18.392673+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:56:50.498549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820014+0000","last_change":"2023-11-24T06:17:17.413669+0000","last_active":"2023-11-24T06:17:18.820014+0000","last_peered":"2023-11-24T06:17:18.820014+0000","last_clean":"2023-11-24T06:17:18.820014+0000","last_became_active":"2023-11-24T06:17:17.413014+0000","last_became_peered":"2023-11-24T06:17:17.413014+0000","last_unstale":"2023-11-24T06:17:18.820014+0000","last_undegraded":"2023-11-24T06:17:18.820014+0000","last_fullsized":"2023-11-24T06:17:18.820014+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:04:51.467527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392341+0000","last_change":"2023-11-24T06:17:16.421306+0000","last_active":"2023-11-24T06:17:18.392341+0000","last_peered":"2023-11-24T06:17:18.392341+0000","last_clean":"2023-11-24T06:17:18.392341+0000","last_became_active":"2023-11-24T06:17:16.420998+0000","last_became_peered":"2023-11-24T06:17:16.420998+0000","last_unstale":"2023-11-24T06:17:18.392341+0000","last_undegraded":"2023-11-24T06:17:18.392341+0000","last_fullsized":"2023-11-24T06:17:18.392341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:07.073655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.816827+0000","last_change":"2023-11-24T06:17:17.415530+0000","last_active":"2023-11-24T06:17:18.816827+0000","last_peered":"2023-11-24T06:17:18.816827+0000","last_clean":"2023-11-24T06:17:18.816827+0000","last_became_active":"2023-11-24T06:17:17.415239+0000","last_became_peered":"2023-11-24T06:17:17.415239+0000","last_unstale":"2023-11-24T06:17:18.816827+0000","last_undegraded":"2023-11-24T06:17:18.816827+0000","last_fullsized":"2023-11-24T06:17:18.816827+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:17:50.847709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392420+0000","last_change":"2023-11-24T06:17:16.421728+0000","last_active":"2023-11-24T06:17:18.392420+0000","last_peered":"2023-11-24T06:17:18.392420+0000","last_clean":"2023-11-24T06:17:18.392420+0000","last_became_active":"2023-11-24T06:17:16.421403+0000","last_became_peered":"2023-11-24T06:17:16.421403+0000","last_unstale":"2023-11-24T06:17:18.392420+0000","last_undegraded":"2023-11-24T06:17:18.392420+0000","last_fullsized":"2023-11-24T06:17:18.392420+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:54:40.137540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.817559+0000","last_change":"2023-11-24T06:17:17.416153+0000","last_active":"2023-11-24T06:17:18.817559+0000","last_peered":"2023-11-24T06:17:18.817559+0000","last_clean":"2023-11-24T06:17:18.817559+0000","last_became_active":"2023-11-24T06:17:17.414853+0000","last_became_peered":"2023-11-24T06:17:17.414853+0000","last_unstale":"2023-11-24T06:17:18.817559+0000","last_undegraded":"2023-11-24T06:17:18.817559+0000","last_fullsized":"2023-11-24T06:17:18.817559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:43:04.857890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.813862+0000","last_change":"2023-11-24T06:17:16.422145+0000","last_active":"2023-11-24T06:17:18.813862+0000","last_peered":"2023-11-24T06:17:18.813862+0000","last_clean":"2023-11-24T06:17:18.813862+0000","last_became_active":"2023-11-24T06:17:16.421869+0000","last_became_peered":"2023-11-24T06:17:16.421869+0000","last_unstale":"2023-11-24T06:17:18.813862+0000","last_undegraded":"2023-11-24T06:17:18.813862+0000","last_fullsized":"2023-11-24T06:17:18.813862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:04:50.466051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.816870+0000","last_change":"2023-11-24T06:17:17.412904+0000","last_active":"2023-11-24T06:17:18.816870+0000","last_peered":"2023-11-24T06:17:18.816870+0000","last_clean":"2023-11-24T06:17:18.816870+0000","last_became_active":"2023-11-24T06:17:17.412575+0000","last_became_peered":"2023-11-24T06:17:17.412575+0000","last_unstale":"2023-11-24T06:17:18.816870+0000","last_undegraded":"2023-11-24T06:17:18.816870+0000","last_fullsized":"2023-11-24T06:17:18.816870+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:50.227692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819989+0000","last_change":"2023-11-24T06:17:16.422042+0000","last_active":"2023-11-24T06:17:18.819989+0000","last_peered":"2023-11-24T06:17:18.819989+0000","last_clean":"2023-11-24T06:17:18.819989+0000","last_became_active":"2023-11-24T06:17:16.421002+0000","last_became_peered":"2023-11-24T06:17:16.421002+0000","last_unstale":"2023-11-24T06:17:18.819989+0000","last_undegraded":"2023-11-24T06:17:18.819989+0000","last_fullsized":"2023-11-24T06:17:18.819989+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:10:22.786967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.816510+0000","last_change":"2023-11-24T06:17:17.416486+0000","last_active":"2023-11-24T06:17:18.816510+0000","last_peered":"2023-11-24T06:17:18.816510+0000","last_clean":"2023-11-24T06:17:18.816510+0000","last_became_active":"2023-11-24T06:17:17.416143+0000","last_became_peered":"2023-11-24T06:17:17.416143+0000","last_unstale":"2023-11-24T06:17:18.816510+0000","last_undegraded":"2023-11-24T06:17:18.816510+0000","last_fullsized":"2023-11-24T06:17:18.816510+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:36.175677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391162+0000","last_change":"2023-11-24T06:17:16.426462+0000","last_active":"2023-11-24T06:17:18.391162+0000","last_peered":"2023-11-24T06:17:18.391162+0000","last_clean":"2023-11-24T06:17:18.391162+0000","last_became_active":"2023-11-24T06:17:16.426184+0000","last_became_peered":"2023-11-24T06:17:16.426184+0000","last_unstale":"2023-11-24T06:17:18.391162+0000","last_undegraded":"2023-11-24T06:17:18.391162+0000","last_fullsized":"2023-11-24T06:17:18.391162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:55:05.040137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.392080+0000","last_change":"2023-11-24T06:17:17.420511+0000","last_active":"2023-11-24T06:17:18.392080+0000","last_peered":"2023-11-24T06:17:18.392080+0000","last_clean":"2023-11-24T06:17:18.392080+0000","last_became_active":"2023-11-24T06:17:17.420133+0000","last_became_peered":"2023-11-24T06:17:17.420133+0000","last_unstale":"2023-11-24T06:17:18.392080+0000","last_undegraded":"2023-11-24T06:17:18.392080+0000","last_fullsized":"2023-11-24T06:17:18.392080+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:49:53.916506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.818785+0000","last_change":"2023-11-24T06:17:16.430044+0000","last_active":"2023-11-24T06:17:18.818785+0000","last_peered":"2023-11-24T06:17:18.818785+0000","last_clean":"2023-11-24T06:17:18.818785+0000","last_became_active":"2023-11-24T06:17:16.429766+0000","last_became_peered":"2023-11-24T06:17:16.429766+0000","last_unstale":"2023-11-24T06:17:18.818785+0000","last_undegraded":"2023-11-24T06:17:18.818785+0000","last_fullsized":"2023-11-24T06:17:18.818785+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:07:47.032762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.816590+0000","last_change":"2023-11-24T06:17:17.422038+0000","last_active":"2023-11-24T06:17:18.816590+0000","last_peered":"2023-11-24T06:17:18.816590+0000","last_clean":"2023-11-24T06:17:18.816590+0000","last_became_active":"2023-11-24T06:17:17.421727+0000","last_became_peered":"2023-11-24T06:17:17.421727+0000","last_unstale":"2023-11-24T06:17:18.816590+0000","last_undegraded":"2023-11-24T06:17:18.816590+0000","last_fullsized":"2023-11-24T06:17:18.816590+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:09.112474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820758+0000","last_change":"2023-11-24T06:17:16.427797+0000","last_active":"2023-11-24T06:17:18.820758+0000","last_peered":"2023-11-24T06:17:18.820758+0000","last_clean":"2023-11-24T06:17:18.820758+0000","last_became_active":"2023-11-24T06:17:16.427478+0000","last_became_peered":"2023-11-24T06:17:16.427478+0000","last_unstale":"2023-11-24T06:17:18.820758+0000","last_undegraded":"2023-11-24T06:17:18.820758+0000","last_fullsized":"2023-11-24T06:17:18.820758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:37:14.478115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.822726+0000","last_change":"2023-11-24T06:17:17.413936+0000","last_active":"2023-11-24T06:17:18.822726+0000","last_peered":"2023-11-24T06:17:18.822726+0000","last_clean":"2023-11-24T06:17:18.822726+0000","last_became_active":"2023-11-24T06:17:17.413529+0000","last_became_peered":"2023-11-24T06:17:17.413529+0000","last_unstale":"2023-11-24T06:17:18.822726+0000","last_undegraded":"2023-11-24T06:17:18.822726+0000","last_fullsized":"2023-11-24T06:17:18.822726+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:41.919440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.815004+0000","last_change":"2023-11-24T06:17:16.422261+0000","last_active":"2023-11-24T06:17:18.815004+0000","last_peered":"2023-11-24T06:17:18.815004+0000","last_clean":"2023-11-24T06:17:18.815004+0000","last_became_active":"2023-11-24T06:17:16.421875+0000","last_became_peered":"2023-11-24T06:17:16.421875+0000","last_unstale":"2023-11-24T06:17:18.815004+0000","last_undegraded":"2023-11-24T06:17:18.815004+0000","last_fullsized":"2023-11-24T06:17:18.815004+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:53:15.915836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.812708+0000","last_change":"2023-11-24T06:17:17.416143+0000","last_active":"2023-11-24T06:17:18.812708+0000","last_peered":"2023-11-24T06:17:18.812708+0000","last_clean":"2023-11-24T06:17:18.812708+0000","last_became_active":"2023-11-24T06:17:17.412289+0000","last_became_peered":"2023-11-24T06:17:17.412289+0000","last_unstale":"2023-11-24T06:17:18.812708+0000","last_undegraded":"2023-11-24T06:17:18.812708+0000","last_fullsized":"2023-11-24T06:17:18.812708+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:59:25.646482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.820177+0000","last_change":"2023-11-24T06:17:16.421114+0000","last_active":"2023-11-24T06:17:18.820177+0000","last_peered":"2023-11-24T06:17:18.820177+0000","last_clean":"2023-11-24T06:17:18.820177+0000","last_became_active":"2023-11-24T06:17:16.419740+0000","last_became_peered":"2023-11-24T06:17:16.419740+0000","last_unstale":"2023-11-24T06:17:18.820177+0000","last_undegraded":"2023-11-24T06:17:18.820177+0000","last_fullsized":"2023-11-24T06:17:18.820177+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:33:29.653709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819771+0000","last_change":"2023-11-24T06:17:17.425941+0000","last_active":"2023-11-24T06:17:18.819771+0000","last_peered":"2023-11-24T06:17:18.819771+0000","last_clean":"2023-11-24T06:17:18.819771+0000","last_became_active":"2023-11-24T06:17:17.421314+0000","last_became_peered":"2023-11-24T06:17:17.421314+0000","last_unstale":"2023-11-24T06:17:18.819771+0000","last_undegraded":"2023-11-24T06:17:18.819771+0000","last_fullsized":"2023-11-24T06:17:18.819771+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:43:44.859171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.814100+0000","last_change":"2023-11-24T06:17:16.421585+0000","last_active":"2023-11-24T06:17:18.814100+0000","last_peered":"2023-11-24T06:17:18.814100+0000","last_clean":"2023-11-24T06:17:18.814100+0000","last_became_active":"2023-11-24T06:17:16.420794+0000","last_became_peered":"2023-11-24T06:17:16.420794+0000","last_unstale":"2023-11-24T06:17:18.814100+0000","last_undegraded":"2023-11-24T06:17:18.814100+0000","last_fullsized":"2023-11-24T06:17:18.814100+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:43:03.842341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.813754+0000","last_change":"2023-11-24T06:17:17.413706+0000","last_active":"2023-11-24T06:17:18.813754+0000","last_peered":"2023-11-24T06:17:18.813754+0000","last_clean":"2023-11-24T06:17:18.813754+0000","last_became_active":"2023-11-24T06:17:17.413366+0000","last_became_peered":"2023-11-24T06:17:17.413366+0000","last_unstale":"2023-11-24T06:17:18.813754+0000","last_undegraded":"2023-11-24T06:17:18.813754+0000","last_fullsized":"2023-11-24T06:17:18.813754+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:15:54.423492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819472+0000","last_change":"2023-11-24T06:17:16.424845+0000","last_active":"2023-11-24T06:17:18.819472+0000","last_peered":"2023-11-24T06:17:18.819472+0000","last_clean":"2023-11-24T06:17:18.819472+0000","last_became_active":"2023-11-24T06:17:16.424563+0000","last_became_peered":"2023-11-24T06:17:16.424563+0000","last_unstale":"2023-11-24T06:17:18.819472+0000","last_undegraded":"2023-11-24T06:17:18.819472+0000","last_fullsized":"2023-11-24T06:17:18.819472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:44:07.073655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.387595+0000","last_change":"2023-11-24T06:17:17.420812+0000","last_active":"2023-11-24T06:17:18.387595+0000","last_peered":"2023-11-24T06:17:18.387595+0000","last_clean":"2023-11-24T06:17:18.387595+0000","last_became_active":"2023-11-24T06:17:17.419365+0000","last_became_peered":"2023-11-24T06:17:17.419365+0000","last_unstale":"2023-11-24T06:17:18.387595+0000","last_undegraded":"2023-11-24T06:17:18.387595+0000","last_fullsized":"2023-11-24T06:17:18.387595+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:39:42.344668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391419+0000","last_change":"2023-11-24T06:17:16.424980+0000","last_active":"2023-11-24T06:17:18.391419+0000","last_peered":"2023-11-24T06:17:18.391419+0000","last_clean":"2023-11-24T06:17:18.391419+0000","last_became_active":"2023-11-24T06:17:16.424673+0000","last_became_peered":"2023-11-24T06:17:16.424673+0000","last_unstale":"2023-11-24T06:17:18.391419+0000","last_undegraded":"2023-11-24T06:17:18.391419+0000","last_fullsized":"2023-11-24T06:17:18.391419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:38.228122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.390926+0000","last_change":"2023-11-24T06:17:17.412862+0000","last_active":"2023-11-24T06:17:18.390926+0000","last_peered":"2023-11-24T06:17:18.390926+0000","last_clean":"2023-11-24T06:17:18.390926+0000","last_became_active":"2023-11-24T06:17:17.412573+0000","last_became_peered":"2023-11-24T06:17:17.412573+0000","last_unstale":"2023-11-24T06:17:18.390926+0000","last_undegraded":"2023-11-24T06:17:18.390926+0000","last_fullsized":"2023-11-24T06:17:18.390926+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T07:50:21.979650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.388048+0000","last_change":"2023-11-24T06:17:16.430350+0000","last_active":"2023-11-24T06:17:18.388048+0000","last_peered":"2023-11-24T06:17:18.388048+0000","last_clean":"2023-11-24T06:17:18.388048+0000","last_became_active":"2023-11-24T06:17:16.429825+0000","last_became_peered":"2023-11-24T06:17:16.429825+0000","last_unstale":"2023-11-24T06:17:18.388048+0000","last_undegraded":"2023-11-24T06:17:18.388048+0000","last_fullsized":"2023-11-24T06:17:18.388048+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:11.836881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.394025+0000","last_change":"2023-11-24T06:17:17.421289+0000","last_active":"2023-11-24T06:17:18.394025+0000","last_peered":"2023-11-24T06:17:18.394025+0000","last_clean":"2023-11-24T06:17:18.394025+0000","last_became_active":"2023-11-24T06:17:17.420149+0000","last_became_peered":"2023-11-24T06:17:17.420149+0000","last_unstale":"2023-11-24T06:17:18.394025+0000","last_undegraded":"2023-11-24T06:17:18.394025+0000","last_fullsized":"2023-11-24T06:17:18.394025+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:16.373614+0000","last_clean_scrub_stamp":"2023-11-24T06:17:16.373614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T06:39:42.344668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.819095+0000","last_change":"2023-11-24T06:17:16.423584+0000","last_active":"2023-11-24T06:17:18.819095+0000","last_peered":"2023-11-24T06:17:18.819095+0000","last_clean":"2023-11-24T06:17:18.819095+0000","last_became_active":"2023-11-24T06:17:16.423260+0000","last_became_peered":"2023-11-24T06:17:16.423260+0000","last_unstale":"2023-11-24T06:17:18.819095+0000","last_undegraded":"2023-11-24T06:17:18.819095+0000","last_fullsized":"2023-11-24T06:17:18.819095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:17:15.358065+0000","last_clean_scrub_stamp":"2023-11-24T06:17:15.358065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:28:07.275105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":174,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-24T06:17:17.408126+0000","last_change":"2023-11-24T06:15:42.021608+0000","last_active":"2023-11-24T06:17:17.408126+0000","last_peered":"2023-11-24T06:17:17.408126+0000","last_clean":"2023-11-24T06:17:17.408126+0000","last_became_active":"2023-11-24T06:15:42.018847+0000","last_became_peered":"2023-11-24T06:15:42.018847+0000","last_unstale":"2023-11-24T06:17:17.408126+0000","last_undegraded":"2023-11-24T06:17:17.408126+0000","last_fullsized":"2023-11-24T06:17:17.408126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_clean_scrub_stamp":"2023-11-24T06:15:40.694368+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:41.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509462,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":987136,"data_stored":744396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.663},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.6359999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.7689999999999999,"5min":1.7689999999999999,"15min":1.7689999999999999},"last":2.1720000000000002},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":2.7719999999999998,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":1.3400000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":2.8820000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":4.0579999999999998,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.778}]},{"osd":3,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":3.4239999999999999,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":1.585},{"interface":"front","average":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":3.0569999999999999,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":1.857}]},{"osd":4,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"min":{"1min":1.016,"5min":1.016,"15min":1.016},"max":{"1min":5.7560000000000002,"5min":5.7560000000000002,"15min":5.7560000000000002},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"min":{"1min":1.018,"5min":1.018,"15min":1.018},"max":{"1min":2.681,"5min":2.681,"15min":2.681},"last":1.724}]},{"osd":5,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":2.117,"5min":2.117,"15min":2.117},"min":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":6.6260000000000003,"5min":6.6260000000000003,"15min":6.6260000000000003},"last":2.0960000000000001},{"interface":"front","average":{"1min":2.1789999999999998,"5min":2.1789999999999998,"15min":2.1789999999999998},"min":{"1min":1.21,"5min":1.21,"15min":1.21},"max":{"1min":6.4169999999999998,"5min":6.4169999999999998,"15min":6.4169999999999998},"last":1.484}]},{"osd":6,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"min":{"1min":1.306,"5min":1.306,"15min":1.306},"max":{"1min":4.4050000000000002,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":1.913},{"interface":"front","average":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"min":{"1min":1.337,"5min":1.337,"15min":1.337},"max":{"1min":3.8540000000000001,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":2.0139999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":397312,"data_stored":154042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.3210000000000002,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.871},{"interface":"front","average":{"1min":1.101,"5min":1.101,"15min":1.101},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.3740000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.8420000000000001,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.2809999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":2.0219999999999998,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.9650000000000001,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.488}]},{"osd":3,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.331,"5min":2.331,"15min":2.331},"last":1.556},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"min":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":1.593}]},{"osd":5,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.631,"5min":1.631,"15min":1.631},"min":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"max":{"1min":2.6989999999999998,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":2.4740000000000002,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.6339999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.702,"5min":1.702,"15min":1.702},"min":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"max":{"1min":2.5339999999999998,"5min":2.5339999999999998,"15min":2.5339999999999998},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.766,"5min":1.766,"15min":1.766},"min":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":2.802,"5min":2.802,"15min":2.802},"last":1.7769999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.788,"5min":1.788,"15min":1.788},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":3.4550000000000001,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":1.708},{"interface":"front","average":{"1min":1.774,"5min":1.774,"15min":1.774},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":2.6949999999999998,"5min":2.6949999999999998,"15min":2.6949999999999998},"last":1.7350000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542166,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":148254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.7410000000000001,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.163,"15min":1.163},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.1779999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.339,"15min":1.339},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.1110000000000002,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.054}]},{"osd":2,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.177,"5min":1.177,"15min":1.177},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.343},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.401}]},{"osd":3,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.409,"15min":1.409},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.601,"15min":1.601},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":3.5249999999999999,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":1.6140000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.6040000000000001,"15min":1.6040000000000001},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":2.7290000000000001,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.3149999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.4670000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.5179999999999998,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":3.1459999999999999,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.1180000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:16:40 2023","interfaces":[{"interface":"back","average":{"1min":1.788,"5min":1.788,"15min":1.788},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.6880000000000002,"5min":2.6880000000000002,"15min":2.6880000000000002},"last":1.819},{"interface":"front","average":{"1min":1.762,"5min":1.762,"15min":1.762},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.6699999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":148310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.657},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.94599999999999995}]},{"osd":1,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.861},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.4390000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.24},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.7609999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.419},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.3860000000000001,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":1.379}]},{"osd":4,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"min":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.542},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.5089999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":1.504}]},{"osd":6,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.5700000000000001,"5min":1.5700000000000001,"15min":1.5700000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.6480000000000001,"5min":2.6480000000000001,"15min":2.6480000000000001},"last":1.294},{"interface":"front","average":{"1min":1.631,"5min":1.631,"15min":1.631},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":3.1560000000000001,"5min":3.1560000000000001,"15min":3.1560000000000001},"last":1.337}]},{"osd":7,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.659,"15min":1.659},"min":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"max":{"1min":3.2949999999999999,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.474},{"interface":"front","average":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":4.1779999999999999,"5min":4.1779999999999999,"15min":4.1779999999999999},"last":1.1679999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":405504,"data_stored":155572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.6699999999999999,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":0.879},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":1.232}]},{"osd":1,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.0249999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.5289999999999999,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.129}]},{"osd":4,"last update":"Fri Nov 24 06:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.1200000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"max":{"1min":2.8279999999999998,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.1719999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.569,"5min":1.569,"15min":1.569},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":3.0089999999999999,"5min":3.0089999999999999,"15min":3.0089999999999999},"last":1.286}]},{"osd":6,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":2.3500000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.377},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"min":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"max":{"1min":2.7370000000000001,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.427}]},{"osd":7,"last update":"Fri Nov 24 06:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.573,"15min":1.573},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":1.079,"5min":1.079,"15min":1.079},"max":{"1min":3.1280000000000001,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.5880000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542167,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":147708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.55900000000000005},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.5390000000000001,"5min":2.5390000000000001,"15min":2.5390000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":1.7849999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.6259999999999999,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":0.57799999999999996}]},{"osd":3,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":0.59599999999999997},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.75,"5min":1.75,"15min":1.75},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.96799999999999997}]},{"osd":5,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.474,"5min":1.474,"15min":1.474},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":2.0350000000000001,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.177}]},{"osd":6,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":3.0499999999999998,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.397,"5min":1.397,"15min":1.397},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":2.9049999999999998,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":0.95399999999999996}]}]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":405504,"data_stored":155572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.92200000000000004}]},{"osd":2,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.3090000000000002,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.1910000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.0940000000000001},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.5619999999999998,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":1.228}]},{"osd":4,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.1659999999999999,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.55}]},{"osd":5,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.153},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":2.355,"5min":2.355,"15min":2.355},"last":1.494}]},{"osd":6,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.3050000000000002,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.385},{"interface":"front","average":{"1min":1.409,"5min":1.409,"15min":1.409},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":2.298,"5min":2.298,"15min":2.298},"last":1.413}]},{"osd":7,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.492,"5min":2.492,"15min":2.492},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.47,"5min":1.47,"15min":1.47},"min":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.4079999999999999,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.637}]}]},{"osd":0,"up_from":14,"seq":60129542166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":966656,"data_stored":737287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":2.7320000000000002,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":0.69699999999999995}]},{"osd":2,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.4870000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":2.5209999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":1.6699999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":1.522}]},{"osd":4,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.254,"5min":2.254,"15min":2.254},"last":1.405},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.347}]},{"osd":5,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.448,"15min":1.448},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.4380000000000002,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.792},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.4830000000000001,"15min":1.4830000000000001},"min":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"max":{"1min":2.3799999999999999,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.603}]},{"osd":6,"last update":"Fri Nov 24 06:16:39 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.526,"15min":1.526},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":2.3159999999999998,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.8540000000000001},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":2.5699999999999998,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.752}]},{"osd":7,"last update":"Fri Nov 24 06:16:43 2023","interfaces":[{"interface":"back","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":2.4620000000000002,"5min":2.4620000000000002,"15min":2.4620000000000002},"last":1.988},{"interface":"front","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":1.002,"5min":1.002,"15min":1.002},"max":{"1min":2.6150000000000002,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.925}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:17:22.758 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T06:17:22.758 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T06:17:23.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.035+0000 7f5a3bfff640 1 Processor -- start 2023-11-24T06:17:23.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.035+0000 7f5a3bfff640 1 -- start start 2023-11-24T06:17:23.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.037+0000 7f5a3bfff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5a3c0d36d0 0x7f5a3c0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.037+0000 7f5a3bfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a3c0b3220 0x7f5a3c0ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.037+0000 7f5a3bfff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a3c160cc0 con 0x7f5a3c1608e0 2023-11-24T06:17:23.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.037+0000 7f5a3bfff640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a3c0d2ab0 con 0x7f5a3c0d36d0 2023-11-24T06:17:23.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.037+0000 7f5a3bfff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0b3a10 con 0x7f5a3c0b3220 2023-11-24T06:17:23.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.037+0000 7f5a3affd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5a3c1608e0 0x7f5a3c0d2fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35862/0 (socket says 172.21.15.5:35862) 2023-11-24T06:17:23.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.037+0000 7f5a3affd640 1 -- 172.21.15.5:0/2354632477 learned_addr learned my addr 172.21.15.5:0/2354632477 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:23.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.037+0000 7f5a3a7fc640 1 --2- 172.21.15.5:0/2354632477 >> v2:172.21.15.6:3300/0 conn(0x7f5a3c0d36d0 0x7f5a3c0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:23.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.037+0000 7f5a39ffb640 1 -- 172.21.15.5:0/2354632477 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638428387 0 0) 0x7f5a3c160cc0 con 0x7f5a3c1608e0 2023-11-24T06:17:23.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.037+0000 7f5a39ffb640 1 -- 172.21.15.5:0/2354632477 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a24003520 con 0x7f5a3c1608e0 2023-11-24T06:17:23.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a3b7fe640 1 --2- 172.21.15.5:0/2354632477 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a3c0b3220 0x7f5a3c0ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:23.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a39ffb640 1 -- 172.21.15.5:0/2354632477 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3179122561 0 0) 0x7f5a24003520 con 0x7f5a3c1608e0 2023-11-24T06:17:23.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a39ffb640 1 -- 172.21.15.5:0/2354632477 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a3c0b3220 msgr2=0x7f5a3c0ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:23.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a39ffb640 1 --2- 172.21.15.5:0/2354632477 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a3c0b3220 0x7f5a3c0ce460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a39ffb640 1 -- 172.21.15.5:0/2354632477 >> v2:172.21.15.6:3300/0 conn(0x7f5a3c0d36d0 msgr2=0x7f5a3c0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:23.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a39ffb640 1 --2- 172.21.15.5:0/2354632477 >> v2:172.21.15.6:3300/0 conn(0x7f5a3c0d36d0 0x7f5a3c0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a39ffb640 1 -- 172.21.15.5:0/2354632477 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c0b3cf0 con 0x7f5a3c1608e0 2023-11-24T06:17:23.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a39ffb640 1 -- 172.21.15.5:0/2354632477 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f5a2c0019f0 con 0x7f5a3c1608e0 2023-11-24T06:17:23.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a39ffb640 1 -- 172.21.15.5:0/2354632477 >> v1:172.21.15.5:6789/0 conn(0x7f5a3c1608e0 legacy=0x7f5a3c0d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:23.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a39ffb640 1 --2- 172.21.15.5:0/2354632477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a240039e0 0x7f5a24003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a39ffb640 1 --2- 172.21.15.5:0/2354632477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a240047c0 0x7f5a24004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a39ffb640 1 --2- 172.21.15.5:0/2354632477 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a3c0b3220 0x7f5a24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a39ffb640 1 -- 172.21.15.5:0/2354632477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0d2ab0 con 0x7f5a240047c0 2023-11-24T06:17:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a39ffb640 1 -- 172.21.15.5:0/2354632477 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0b3a10 con 0x7f5a240039e0 2023-11-24T06:17:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a39ffb640 1 -- 172.21.15.5:0/2354632477 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2400b980 con 0x7f5a3c0b3220 2023-11-24T06:17:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.038+0000 7f5a3a7fc640 1 --2- 172.21.15.5:0/2354632477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a240047c0 0x7f5a24004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.039+0000 7f5a3a7fc640 1 -- 172.21.15.5:0/2354632477 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a3c0b3220 msgr2=0x7f5a24005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.039+0000 7f5a3a7fc640 1 --2- 172.21.15.5:0/2354632477 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a3c0b3220 0x7f5a24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.039+0000 7f5a3a7fc640 1 -- 172.21.15.5:0/2354632477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a240039e0 msgr2=0x7f5a24003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.039+0000 7f5a3a7fc640 1 --2- 172.21.15.5:0/2354632477 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a240039e0 0x7f5a24003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.039+0000 7f5a3a7fc640 1 -- 172.21.15.5:0/2354632477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a24003520 con 0x7f5a240047c0 2023-11-24T06:17:23.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.040+0000 7f5a3a7fc640 1 --2- 172.21.15.5:0/2354632477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a240047c0 0x7f5a24004c40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009af0 tx=0x7f5a240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1d6cefa5431a80 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:23.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.040+0000 7f5a39ffb640 1 -- 172.21.15.5:0/2354632477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3002c710 con 0x7f5a240047c0 2023-11-24T06:17:23.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.040+0000 7f5a39ffb640 1 -- 172.21.15.5:0/2354632477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3002c870 con 0x7f5a240047c0 2023-11-24T06:17:23.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.040+0000 7f5a39ffb640 1 -- 172.21.15.5:0/2354632477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3002cc00 con 0x7f5a240047c0 2023-11-24T06:17:23.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.041+0000 7f5a3bfff640 1 -- 172.21.15.5:0/2354632477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a240047c0 msgr2=0x7f5a24004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:23.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.041+0000 7f5a3bfff640 1 --2- 172.21.15.5:0/2354632477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a240047c0 0x7f5a24004c40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009af0 tx=0x7f5a240057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.041+0000 7f5a3b7fe640 1 -- 172.21.15.5:0/2354632477 reap_dead start 2023-11-24T06:17:23.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.041+0000 7f5a3bfff640 1 -- 172.21.15.5:0/2354632477 shutdown_connections 2023-11-24T06:17:23.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.041+0000 7f5a3bfff640 1 -- 172.21.15.5:0/2354632477 >> 172.21.15.5:0/2354632477 conn(0x7f5a3c066b70 msgr2=0x7f5a3c09c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:23.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.041+0000 7f5a3bfff640 1 -- 172.21.15.5:0/2354632477 shutdown_connections 2023-11-24T06:17:23.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.041+0000 7f5a3bfff640 1 -- 172.21.15.5:0/2354632477 wait complete. 2023-11-24T06:17:23.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.043+0000 7f5a3bfff640 1 Processor -- start 2023-11-24T06:17:23.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.043+0000 7f5a3bfff640 1 -- start start 2023-11-24T06:17:23.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.044+0000 7f5a3bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a240047c0 0x7f5a3c0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.044+0000 7f5a3bfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a3c1546b0 0x7f5a3c1f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.044+0000 7f5a3bfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a3c0b6e50 0x7f5a3c0cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.044+0000 7f5a3bfff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0b5d10 con 0x7f5a240047c0 2023-11-24T06:17:23.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.044+0000 7f5a3bfff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c1f35d0 con 0x7f5a3c1546b0 2023-11-24T06:17:23.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.044+0000 7f5a3bfff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c1f38b0 con 0x7f5a3c0b6e50 2023-11-24T06:17:23.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.044+0000 7f5a3affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a240047c0 0x7f5a3c0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:23.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.044+0000 7f5a3affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a240047c0 0x7f5a3c0cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37396/0 (socket says 172.21.15.5:37396) 2023-11-24T06:17:23.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.044+0000 7f5a3affd640 1 -- 172.21.15.5:0/1886394613 learned_addr learned my addr 172.21.15.5:0/1886394613 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:23.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.045+0000 7f5a3a7fc640 1 --2- 172.21.15.5:0/1886394613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a3c1546b0 0x7f5a3c1f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:23.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.045+0000 7f5a3affd640 1 -- 172.21.15.5:0/1886394613 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a3c0b6e50 msgr2=0x7f5a3c0cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:23.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.045+0000 7f5a3affd640 1 --2- 172.21.15.5:0/1886394613 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a3c0b6e50 0x7f5a3c0cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.045+0000 7f5a3affd640 1 -- 172.21.15.5:0/1886394613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a3c1546b0 msgr2=0x7f5a3c1f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:23.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.045+0000 7f5a3affd640 1 --2- 172.21.15.5:0/1886394613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a3c1546b0 0x7f5a3c1f2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.045+0000 7f5a3affd640 1 -- 172.21.15.5:0/1886394613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c1f3b90 con 0x7f5a240047c0 2023-11-24T06:17:23.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.045+0000 7f5a3affd640 1 --2- 172.21.15.5:0/1886394613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a240047c0 0x7f5a3c0cc730 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00cac0 tx=0x7f5a2c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:23.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.045+0000 7f5a237fe640 1 -- 172.21.15.5:0/1886394613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c00a7a0 con 0x7f5a240047c0 2023-11-24T06:17:23.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.045+0000 7f5a237fe640 1 -- 172.21.15.5:0/1886394613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2c00a900 con 0x7f5a240047c0 2023-11-24T06:17:23.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.045+0000 7f5a237fe640 1 -- 172.21.15.5:0/1886394613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c00acb0 con 0x7f5a240047c0 2023-11-24T06:17:23.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.045+0000 7f5a3bfff640 1 -- 172.21.15.5:0/1886394613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3c1f3e70 con 0x7f5a240047c0 2023-11-24T06:17:23.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.045+0000 7f5a3bfff640 1 -- 172.21.15.5:0/1886394613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c0b7300 con 0x7f5a240047c0 2023-11-24T06:17:23.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.046+0000 7f5a3bfff640 1 -- 172.21.15.5:0/1886394613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3c0b8fc0 con 0x7f5a240047c0 2023-11-24T06:17:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.047+0000 7f5a237fe640 1 -- 172.21.15.5:0/1886394613 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a2c0070d0 con 0x7f5a240047c0 2023-11-24T06:17:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.048+0000 7f5a237fe640 1 --2- 172.21.15.5:0/1886394613 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5a2806ae70 0x7f5a2806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.048+0000 7f5a237fe640 1 -- 172.21.15.5:0/1886394613 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a2c088d00 con 0x7f5a240047c0 2023-11-24T06:17:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.049+0000 7f5a3a7fc640 1 --2- 172.21.15.5:0/1886394613 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5a2806ae70 0x7f5a2806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:23.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.049+0000 7f5a3a7fc640 1 --2- 172.21.15.5:0/1886394613 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5a2806ae70 0x7f5a2806d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5a30036000 tx=0x7f5a30030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:23.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.053+0000 7f5a237fe640 1 -- 172.21.15.5:0/1886394613 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a2c053db0 con 0x7f5a240047c0 2023-11-24T06:17:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.315+0000 7f5a3bfff640 1 -- 172.21.15.5:0/1886394613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5a3c154a90 con 0x7f5a240047c0 2023-11-24T06:17:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.319+0000 7f5a237fe640 1 -- 172.21.15.5:0/1886394613 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5a2c018020 con 0x7f5a240047c0 2023-11-24T06:17:23.323 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2023-11-24T06:17:23.323 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBSP2Bl7ShkEhAAJ7REPm4GEzxnc5jqGBmDlQ== 2023-11-24T06:17:23.323 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2023-11-24T06:17:23.323 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2023-11-24T06:17:23.323 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T06:17:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2023-11-24T06:17:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.322+0000 7f5a3bfff640 1 -- 172.21.15.5:0/1886394613 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5a2806ae70 msgr2=0x7f5a2806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.322+0000 7f5a3bfff640 1 --2- 172.21.15.5:0/1886394613 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5a2806ae70 0x7f5a2806d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5a30036000 tx=0x7f5a30030000 comp rx=0 tx=0).stop 2023-11-24T06:17:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.322+0000 7f5a3bfff640 1 -- 172.21.15.5:0/1886394613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a240047c0 msgr2=0x7f5a3c0cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.322+0000 7f5a3bfff640 1 --2- 172.21.15.5:0/1886394613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a240047c0 0x7f5a3c0cc730 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00cac0 tx=0x7f5a2c00cf80 comp rx=0 tx=0).stop 2023-11-24T06:17:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.322+0000 7f5a3bfff640 1 -- 172.21.15.5:0/1886394613 shutdown_connections 2023-11-24T06:17:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.322+0000 7f5a3bfff640 1 --2- 172.21.15.5:0/1886394613 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5a2806ae70 0x7f5a2806d330 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.322+0000 7f5a3bfff640 1 --2- 172.21.15.5:0/1886394613 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5a3c0b6e50 0x7f5a3c0cf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.322+0000 7f5a3bfff640 1 --2- 172.21.15.5:0/1886394613 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5a3c1546b0 0x7f5a3c1f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.322+0000 7f5a3bfff640 1 --2- 172.21.15.5:0/1886394613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a240047c0 0x7f5a3c0cc730 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.322+0000 7f5a3bfff640 1 -- 172.21.15.5:0/1886394613 >> 172.21.15.5:0/1886394613 conn(0x7f5a3c066b70 msgr2=0x7f5a3c15b960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.323+0000 7f5a3bfff640 1 -- 172.21.15.5:0/1886394613 shutdown_connections 2023-11-24T06:17:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.323+0000 7f5a3bfff640 1 -- 172.21.15.5:0/1886394613 wait complete. 2023-11-24T06:17:23.344 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:17:23.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.563+0000 7fa1162d1640 1 Processor -- start 2023-11-24T06:17:23.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.563+0000 7fa1162d1640 1 -- start start 2023-11-24T06:17:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.565+0000 7fa1162d1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa110157770 0x7fa11006cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.565+0000 7fa1162d1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa11006d080 0x7fa11015d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.565+0000 7fa1162d1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa110161670 con 0x7fa11015ddb0 2023-11-24T06:17:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.565+0000 7fa1162d1640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11006d500 con 0x7fa110157770 2023-11-24T06:17:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.565+0000 7fa1162d1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa110071730 con 0x7fa11006d080 2023-11-24T06:17:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.565+0000 7fa1152cf640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa110157770 0x7fa11006cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:23.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.565+0000 7fa1152cf640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa110157770 0x7fa11006cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37412/0 (socket says 172.21.15.5:37412) 2023-11-24T06:17:23.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.565+0000 7fa1152cf640 1 -- 172.21.15.5:0/3095104519 learned_addr learned my addr 172.21.15.5:0/3095104519 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:23.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.566+0000 7fa1152cf640 1 -- 172.21.15.5:0/3095104519 >> v2:172.21.15.6:3300/0 conn(0x7fa11006d080 msgr2=0x7fa11015d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:23.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.566+0000 7fa1152cf640 1 --2- 172.21.15.5:0/3095104519 >> v2:172.21.15.6:3300/0 conn(0x7fa11006d080 0x7fa11015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.566+0000 7fa1152cf640 1 -- 172.21.15.5:0/3095104519 >> v1:172.21.15.6:6789/0 conn(0x7fa11015ddb0 legacy=0x7fa1100ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:23.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.566+0000 7fa1152cf640 1 -- 172.21.15.5:0/3095104519 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11015e610 con 0x7fa110157770 2023-11-24T06:17:23.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.567+0000 7fa1152cf640 1 --2- 172.21.15.5:0/3095104519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa110157770 0x7fa11006cb40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa1100cfcd0 tx=0x7fa10c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccff05d1c1e7475 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:23.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.567+0000 7fa107fff640 1 -- 172.21.15.5:0/3095104519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c031870 con 0x7fa110157770 2023-11-24T06:17:23.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.567+0000 7fa107fff640 1 -- 172.21.15.5:0/3095104519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10c0319d0 con 0x7fa110157770 2023-11-24T06:17:23.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.567+0000 7fa107fff640 1 -- 172.21.15.5:0/3095104519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c031d60 con 0x7fa110157770 2023-11-24T06:17:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.568+0000 7fa1162d1640 1 -- 172.21.15.5:0/3095104519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa110157770 msgr2=0x7fa11006cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.568+0000 7fa1162d1640 1 --2- 172.21.15.5:0/3095104519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa110157770 0x7fa11006cb40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa1100cfcd0 tx=0x7fa10c02b910 comp rx=0 tx=0).stop 2023-11-24T06:17:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.568+0000 7fa1162d1640 1 -- 172.21.15.5:0/3095104519 shutdown_connections 2023-11-24T06:17:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.568+0000 7fa1162d1640 1 --2- 172.21.15.5:0/3095104519 >> v2:172.21.15.6:3300/0 conn(0x7fa11006d080 0x7fa11015d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.568+0000 7fa1162d1640 1 --2- 172.21.15.5:0/3095104519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa110157770 0x7fa11006cb40 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.568+0000 7fa1162d1640 1 -- 172.21.15.5:0/3095104519 >> 172.21.15.5:0/3095104519 conn(0x7fa11006a860 msgr2=0x7fa11006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.568+0000 7fa1162d1640 1 -- 172.21.15.5:0/3095104519 shutdown_connections 2023-11-24T06:17:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.568+0000 7fa1162d1640 1 -- 172.21.15.5:0/3095104519 wait complete. 2023-11-24T06:17:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.570+0000 7fa1162d1640 1 Processor -- start 2023-11-24T06:17:23.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.570+0000 7fa1162d1640 1 -- start start 2023-11-24T06:17:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.571+0000 7fa1162d1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa11006d080 0x7fa1101f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.571+0000 7fa1162d1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa110157770 0x7fa110144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.571+0000 7fa1162d1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa11015ddb0 0x7fa1101f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.571+0000 7fa1162d1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1101f6330 con 0x7fa110157770 2023-11-24T06:17:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.571+0000 7fa1162d1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa110151db0 con 0x7fa11006d080 2023-11-24T06:17:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.571+0000 7fa1162d1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1101f68b0 con 0x7fa11015ddb0 2023-11-24T06:17:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.572+0000 7fa114ace640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa110157770 0x7fa110144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.572+0000 7fa114ace640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa110157770 0x7fa110144030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37424/0 (socket says 172.21.15.5:37424) 2023-11-24T06:17:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.572+0000 7fa114ace640 1 -- 172.21.15.5:0/513322631 learned_addr learned my addr 172.21.15.5:0/513322631 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.572+0000 7fa1152cf640 1 --2- 172.21.15.5:0/513322631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa11006d080 0x7fa1101f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.572+0000 7fa115ad0640 1 --2- 172.21.15.5:0/513322631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa11015ddb0 0x7fa1101f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.572+0000 7fa114ace640 1 -- 172.21.15.5:0/513322631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa11015ddb0 msgr2=0x7fa1101f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.572+0000 7fa114ace640 1 --2- 172.21.15.5:0/513322631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa11015ddb0 0x7fa1101f2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.572+0000 7fa114ace640 1 -- 172.21.15.5:0/513322631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa11006d080 msgr2=0x7fa1101f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.572+0000 7fa114ace640 1 --2- 172.21.15.5:0/513322631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa11006d080 0x7fa1101f6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.572+0000 7fa114ace640 1 -- 172.21.15.5:0/513322631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c0094d0 con 0x7fa110157770 2023-11-24T06:17:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.573+0000 7fa114ace640 1 --2- 172.21.15.5:0/513322631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa110157770 0x7fa110144030 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa10000b440 tx=0x7fa10000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.573+0000 7fa105ffb640 1 -- 172.21.15.5:0/513322631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100011830 con 0x7fa110157770 2023-11-24T06:17:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.573+0000 7fa105ffb640 1 -- 172.21.15.5:0/513322631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa100011990 con 0x7fa110157770 2023-11-24T06:17:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.573+0000 7fa105ffb640 1 -- 172.21.15.5:0/513322631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa100011d40 con 0x7fa110157770 2023-11-24T06:17:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.573+0000 7fa1162d1640 1 -- 172.21.15.5:0/513322631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1101f7280 con 0x7fa110157770 2023-11-24T06:17:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.573+0000 7fa1162d1640 1 -- 172.21.15.5:0/513322631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1101f3850 con 0x7fa110157770 2023-11-24T06:17:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.574+0000 7fa1162d1640 1 -- 172.21.15.5:0/513322631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11004e840 con 0x7fa110157770 2023-11-24T06:17:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.575+0000 7fa105ffb640 1 -- 172.21.15.5:0/513322631 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa10001a400 con 0x7fa110157770 2023-11-24T06:17:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.576+0000 7fa105ffb640 1 --2- 172.21.15.5:0/513322631 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa0f406ae40 0x7fa0f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.576+0000 7fa105ffb640 1 -- 172.21.15.5:0/513322631 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1000897c0 con 0x7fa110157770 2023-11-24T06:17:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.577+0000 7fa1152cf640 1 --2- 172.21.15.5:0/513322631 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa0f406ae40 0x7fa0f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:23.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.577+0000 7fa1152cf640 1 --2- 172.21.15.5:0/513322631 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa0f406ae40 0x7fa0f406d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa10c036000 tx=0x7fa10c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:23.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.581+0000 7fa105ffb640 1 -- 172.21.15.5:0/513322631 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1000547d0 con 0x7fa110157770 2023-11-24T06:17:23.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.808+0000 7fa1162d1640 1 -- 172.21.15.5:0/513322631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa110150fc0 con 0x7fa110157770 2023-11-24T06:17:23.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.809+0000 7fa105ffb640 1 -- 172.21.15.5:0/513322631 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4901 (secure 0 0 0) 0x7fa100015090 con 0x7fa110157770 2023-11-24T06:17:23.813 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:23.813 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6833/2072679798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2072679798},{"type":"v1","addr":"172.21.15.6:6833","nonce":2072679798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3855353235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3855353235},{"type":"v1","addr":"172.21.15.6:6835","nonce":3855353235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2394562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2394562290},{"type":"v1","addr":"172.21.15.5:6836","nonce":2394562290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:17:16.874899+0000","modified":"2023-11-24T06:17:20.326601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5217":{"gid":5217,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/3690265514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3690265514},{"type":"v1","addr":"172.21.15.5:6837","nonce":3690265514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:17:23.813 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2023-11-24T06:17:23.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.812+0000 7fa1162d1640 1 -- 172.21.15.5:0/513322631 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa0f406ae40 msgr2=0x7fa0f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.812+0000 7fa1162d1640 1 --2- 172.21.15.5:0/513322631 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa0f406ae40 0x7fa0f406d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa10c036000 tx=0x7fa10c03c040 comp rx=0 tx=0).stop 2023-11-24T06:17:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.812+0000 7fa1162d1640 1 -- 172.21.15.5:0/513322631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa110157770 msgr2=0x7fa110144030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.812+0000 7fa1162d1640 1 --2- 172.21.15.5:0/513322631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa110157770 0x7fa110144030 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa10000b440 tx=0x7fa10000b900 comp rx=0 tx=0).stop 2023-11-24T06:17:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.812+0000 7fa1162d1640 1 -- 172.21.15.5:0/513322631 shutdown_connections 2023-11-24T06:17:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.812+0000 7fa1162d1640 1 --2- 172.21.15.5:0/513322631 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa0f406ae40 0x7fa0f406d300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.812+0000 7fa1162d1640 1 --2- 172.21.15.5:0/513322631 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa11015ddb0 0x7fa1101f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.812+0000 7fa1162d1640 1 --2- 172.21.15.5:0/513322631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa110157770 0x7fa110144030 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.812+0000 7fa1162d1640 1 --2- 172.21.15.5:0/513322631 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa11006d080 0x7fa1101f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.812+0000 7fa1162d1640 1 -- 172.21.15.5:0/513322631 >> 172.21.15.5:0/513322631 conn(0x7fa11006a860 msgr2=0x7fa110155370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.812+0000 7fa1162d1640 1 -- 172.21.15.5:0/513322631 shutdown_connections 2023-11-24T06:17:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:23.812+0000 7fa1162d1640 1 -- 172.21.15.5:0/513322631 wait complete. 2023-11-24T06:17:24.235 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T06:17:16.874899+0000', 'modified': '2023-11-24T06:17:20.326601+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5217}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5217': {'gid': 5217, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.5:6837/3690265514', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6835', 'nonce': 3690265514}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 3690265514}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T06:17:24.236 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T06:17:24.236 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T06:17:24.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.474+0000 7f81d84ad640 1 Processor -- start 2023-11-24T06:17:24.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.475+0000 7f81d84ad640 1 -- start start 2023-11-24T06:17:24.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.476+0000 7f81d84ad640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f81d015d4f0 0x7f81d00b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.476+0000 7f81d84ad640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d015d8d0 con 0x7f81d00b73c0 2023-11-24T06:17:24.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.476+0000 7f81d84ad640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d00b7900 con 0x7f81d00ce6e0 2023-11-24T06:17:24.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.476+0000 7f81d84ad640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d00b7be0 con 0x7f81d015d4f0 2023-11-24T06:17:24.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.476+0000 7f81d5a48640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f81d00b73c0 0x7f81d00cdfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35870/0 (socket says 172.21.15.5:35870) 2023-11-24T06:17:24.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.476+0000 7f81d5a48640 1 -- 172.21.15.5:0/477278465 learned_addr learned my addr 172.21.15.5:0/477278465 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.477+0000 7f81d6249640 1 --2- 172.21.15.5:0/477278465 >> v2:172.21.15.6:3300/0 conn(0x7f81d015d4f0 0x7f81d00b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.477+0000 7f81d5247640 1 -- 172.21.15.5:0/477278465 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951472450 0 0) 0x7f81d015d8d0 con 0x7f81d00b73c0 2023-11-24T06:17:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.477+0000 7f81d5247640 1 -- 172.21.15.5:0/477278465 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81bc003520 con 0x7f81d00b73c0 2023-11-24T06:17:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.477+0000 7f81d5247640 1 -- 172.21.15.5:0/477278465 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2199761026 0 0) 0x7f81bc003520 con 0x7f81d00b73c0 2023-11-24T06:17:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.477+0000 7f81d5247640 1 -- 172.21.15.5:0/477278465 >> v2:172.21.15.6:3300/0 conn(0x7f81d015d4f0 msgr2=0x7f81d00b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.477+0000 7f81d5247640 1 --2- 172.21.15.5:0/477278465 >> v2:172.21.15.6:3300/0 conn(0x7f81d015d4f0 0x7f81d00b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.477+0000 7f81d5247640 1 -- 172.21.15.5:0/477278465 >> v1:172.21.15.6:6789/0 conn(0x7f81d00ce6e0 legacy=0x7f81d00d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.477+0000 7f81d5247640 1 -- 172.21.15.5:0/477278465 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d00b7ec0 con 0x7f81d00b73c0 2023-11-24T06:17:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.477+0000 7f81d5247640 1 -- 172.21.15.5:0/477278465 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f81c0001690 con 0x7f81d00b73c0 2023-11-24T06:17:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.477+0000 7f81d6249640 1 --2- 172.21.15.5:0/477278465 >> v2:172.21.15.6:3300/0 conn(0x7f81d015d4f0 0x7f81d00b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:17:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.477+0000 7f81d5247640 1 -- 172.21.15.5:0/477278465 >> v1:172.21.15.5:6789/0 conn(0x7f81d00b73c0 legacy=0x7f81d00cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.478+0000 7f81d5247640 1 --2- 172.21.15.5:0/477278465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81bc003970 0x7f81bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.478+0000 7f81d5247640 1 --2- 172.21.15.5:0/477278465 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81bc0047c0 0x7f81bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.478+0000 7f81d5247640 1 --2- 172.21.15.5:0/477278465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc005180 0x7f81bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.478+0000 7f81d5247640 1 -- 172.21.15.5:0/477278465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d00b7be0 con 0x7f81bc005180 2023-11-24T06:17:24.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.478+0000 7f81d5247640 1 -- 172.21.15.5:0/477278465 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc00c0b0 con 0x7f81bc003970 2023-11-24T06:17:24.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.478+0000 7f81d5247640 1 -- 172.21.15.5:0/477278465 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81bc00c270 con 0x7f81bc0047c0 2023-11-24T06:17:24.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.478+0000 7f81d6a4a640 1 --2- 172.21.15.5:0/477278465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc005180 0x7f81bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.478+0000 7f81d6249640 1 --2- 172.21.15.5:0/477278465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81bc003970 0x7f81bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.478+0000 7f81d5a48640 1 --2- 172.21.15.5:0/477278465 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81bc0047c0 0x7f81bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.478+0000 7f81d6a4a640 1 -- 172.21.15.5:0/477278465 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81bc0047c0 msgr2=0x7f81bc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:24.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.478+0000 7f81d6a4a640 1 --2- 172.21.15.5:0/477278465 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81bc0047c0 0x7f81bc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.478+0000 7f81d6a4a640 1 -- 172.21.15.5:0/477278465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81bc003970 msgr2=0x7f81bc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:24.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.478+0000 7f81d6a4a640 1 --2- 172.21.15.5:0/477278465 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81bc003970 0x7f81bc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.478+0000 7f81d6a4a640 1 -- 172.21.15.5:0/477278465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc003520 con 0x7f81bc005180 2023-11-24T06:17:24.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.479+0000 7f81d6a4a640 1 -- 172.21.15.5:0/477278465 reap_dead start 2023-11-24T06:17:24.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.479+0000 7f81d6a4a640 1 --2- 172.21.15.5:0/477278465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc005180 0x7f81bc00b980 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f81c8001e50 tx=0x7f81bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24fd21b0399e49c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:24.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.480+0000 7f81d5247640 1 -- 172.21.15.5:0/477278465 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c8003b70 con 0x7f81bc005180 2023-11-24T06:17:24.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.480+0000 7f81d5247640 1 -- 172.21.15.5:0/477278465 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c8003d10 con 0x7f81bc005180 2023-11-24T06:17:24.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.480+0000 7f81d5247640 1 -- 172.21.15.5:0/477278465 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c8005310 con 0x7f81bc005180 2023-11-24T06:17:24.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.480+0000 7f81d84ad640 1 -- 172.21.15.5:0/477278465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc005180 msgr2=0x7f81bc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.480+0000 7f81d84ad640 1 --2- 172.21.15.5:0/477278465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc005180 0x7f81bc00b980 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f81c8001e50 tx=0x7f81bc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.480+0000 7f81d84ad640 1 -- 172.21.15.5:0/477278465 shutdown_connections 2023-11-24T06:17:24.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.480+0000 7f81d84ad640 1 --2- 172.21.15.5:0/477278465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc005180 0x7f81bc00b980 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.480+0000 7f81d84ad640 1 -- 172.21.15.5:0/477278465 >> 172.21.15.5:0/477278465 conn(0x7f81d0071f90 msgr2=0x7f81d00723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:24.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.481+0000 7f81d84ad640 1 -- 172.21.15.5:0/477278465 shutdown_connections 2023-11-24T06:17:24.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.481+0000 7f81d84ad640 1 -- 172.21.15.5:0/477278465 wait complete. 2023-11-24T06:17:24.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.482+0000 7f81d84ad640 1 Processor -- start 2023-11-24T06:17:24.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.482+0000 7f81d84ad640 1 -- start start 2023-11-24T06:17:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.483+0000 7f81d84ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc005180 0x7f81d0151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.483+0000 7f81d84ad640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81d00cca90 0x7f81d00ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.483+0000 7f81d84ad640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81d01f6450 0x7f81d01f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.484+0000 7f81d84ad640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d01f6e10 con 0x7f81bc005180 2023-11-24T06:17:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.484+0000 7f81d84ad640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d00ca820 con 0x7f81d00cca90 2023-11-24T06:17:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.484+0000 7f81d84ad640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d00cab00 con 0x7f81d01f6450 2023-11-24T06:17:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.484+0000 7f81d6249640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc005180 0x7f81d0151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.484+0000 7f81d6249640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc005180 0x7f81d0151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37446/0 (socket says 172.21.15.5:37446) 2023-11-24T06:17:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.484+0000 7f81d6249640 1 -- 172.21.15.5:0/1599019295 learned_addr learned my addr 172.21.15.5:0/1599019295 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.484+0000 7f81d6249640 1 -- 172.21.15.5:0/1599019295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81d01f6450 msgr2=0x7f81d01f68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.484+0000 7f81d5a48640 1 --2- 172.21.15.5:0/1599019295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81d00cca90 0x7f81d00ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.484+0000 7f81d6249640 1 --2- 172.21.15.5:0/1599019295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81d01f6450 0x7f81d01f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.484+0000 7f81d6249640 1 -- 172.21.15.5:0/1599019295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81d00cca90 msgr2=0x7f81d00ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.484+0000 7f81d6249640 1 --2- 172.21.15.5:0/1599019295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81d00cca90 0x7f81d00ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.484+0000 7f81d6249640 1 -- 172.21.15.5:0/1599019295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d00cade0 con 0x7f81bc005180 2023-11-24T06:17:24.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.485+0000 7f81d6249640 1 --2- 172.21.15.5:0/1599019295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc005180 0x7f81d0151330 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f81cc010860 tx=0x7f81cc010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:24.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.485+0000 7f81c6ffd640 1 -- 172.21.15.5:0/1599019295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00cb40 con 0x7f81bc005180 2023-11-24T06:17:24.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.485+0000 7f81c6ffd640 1 -- 172.21.15.5:0/1599019295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc00cca0 con 0x7f81bc005180 2023-11-24T06:17:24.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.485+0000 7f81d84ad640 1 -- 172.21.15.5:0/1599019295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d00cb0c0 con 0x7f81bc005180 2023-11-24T06:17:24.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.485+0000 7f81c6ffd640 1 -- 172.21.15.5:0/1599019295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc0156b0 con 0x7f81bc005180 2023-11-24T06:17:24.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.485+0000 7f81d84ad640 1 -- 172.21.15.5:0/1599019295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d00ca3d0 con 0x7f81bc005180 2023-11-24T06:17:24.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.487+0000 7f81c6ffd640 1 -- 172.21.15.5:0/1599019295 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f81cc015810 con 0x7f81bc005180 2023-11-24T06:17:24.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.488+0000 7f81c6ffd640 1 --2- 172.21.15.5:0/1599019295 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f81b406ae70 0x7f81b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.488+0000 7f81c6ffd640 1 -- 172.21.15.5:0/1599019295 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81cc089df0 con 0x7f81bc005180 2023-11-24T06:17:24.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.488+0000 7f81d5a48640 1 --2- 172.21.15.5:0/1599019295 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f81b406ae70 0x7f81b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.489+0000 7f81db042740 1 Processor -- start 2023-11-24T06:17:24.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.489+0000 7f81db042740 1 -- start start 2023-11-24T06:17:24.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.489+0000 7f81d5a48640 1 --2- 172.21.15.5:0/1599019295 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f81b406ae70 0x7f81b406d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f81c0001900 tx=0x7f81c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:24.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.490+0000 7f81db042740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f6ea0 0x557d510f7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.490+0000 7f81db042740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 0x557d510f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.490+0000 7f81db042740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f65e0 0x557d5113a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.490+0000 7f81db042740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d51055880 con 0x557d510f6ea0 2023-11-24T06:17:24.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.490+0000 7f81db042740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d51064ae0 con 0x557d510f1ba0 2023-11-24T06:17:24.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.490+0000 7f81db042740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d510621d0 con 0x557d510f65e0 2023-11-24T06:17:24.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.490+0000 7f81d6249640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f6ea0 0x557d510f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.490+0000 7f81d6249640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f6ea0 0x557d510f7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37458/0 (socket says 172.21.15.5:37458) 2023-11-24T06:17:24.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.490+0000 7f81d6249640 1 -- 172.21.15.5:0/1437051207 learned_addr learned my addr 172.21.15.5:0/1437051207 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:24.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.491+0000 7f81d6249640 1 -- 172.21.15.5:0/1437051207 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f65e0 msgr2=0x557d5113a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:24.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.491+0000 7f81d5a48640 1 --2- 172.21.15.5:0/1437051207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 0x557d510f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.491+0000 7f81d6249640 1 --2- 172.21.15.5:0/1437051207 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f65e0 0x557d5113a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.491+0000 7f81d6249640 1 -- 172.21.15.5:0/1437051207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 msgr2=0x557d510f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:24.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.491+0000 7f81d6249640 1 --2- 172.21.15.5:0/1437051207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 0x557d510f60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.491+0000 7f81d6249640 1 -- 172.21.15.5:0/1437051207 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d51025860 con 0x557d510f6ea0 2023-11-24T06:17:24.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.491+0000 7f81d6249640 1 --2- 172.21.15.5:0/1437051207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f6ea0 0x557d510f7280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f81cc054fc0 tx=0x7f81cc022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386966f83e29aa7f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:24.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.491+0000 7f81b3fff640 1 -- 172.21.15.5:0/1437051207 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc05cc10 con 0x557d510f6ea0 2023-11-24T06:17:24.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.491+0000 7f81b3fff640 1 -- 172.21.15.5:0/1437051207 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc060dc0 con 0x557d510f6ea0 2023-11-24T06:17:24.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.491+0000 7f81b3fff640 1 -- 172.21.15.5:0/1437051207 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00ce10 con 0x557d510f6ea0 2023-11-24T06:17:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.492+0000 7f81db042740 1 -- 172.21.15.5:0/1437051207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f6ea0 msgr2=0x557d510f7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.492+0000 7f81db042740 1 --2- 172.21.15.5:0/1437051207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f6ea0 0x557d510f7280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f81cc054fc0 tx=0x7f81cc022000 comp rx=0 tx=0).stop 2023-11-24T06:17:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.492+0000 7f81db042740 1 -- 172.21.15.5:0/1437051207 shutdown_connections 2023-11-24T06:17:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.492+0000 7f81db042740 1 --2- 172.21.15.5:0/1437051207 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f65e0 0x557d5113a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.492+0000 7f81db042740 1 --2- 172.21.15.5:0/1437051207 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 0x557d510f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.492+0000 7f81db042740 1 --2- 172.21.15.5:0/1437051207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f6ea0 0x557d510f7280 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.492+0000 7f81db042740 1 -- 172.21.15.5:0/1437051207 >> 172.21.15.5:0/1437051207 conn(0x557d510b8c70 msgr2=0x557d5113c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.492+0000 7f81db042740 1 -- 172.21.15.5:0/1437051207 shutdown_connections 2023-11-24T06:17:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.492+0000 7f81db042740 1 -- 172.21.15.5:0/1437051207 wait complete. 2023-11-24T06:17:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.493+0000 7f81db042740 1 -- start start 2023-11-24T06:17:24.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.493+0000 7f81db042740 1 Processor -- start 2023-11-24T06:17:24.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.494+0000 7f81db042740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 0x557d5111a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.494+0000 7f81db042740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f65e0 0x557d5111fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.494+0000 7f81db042740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f6ea0 0x557d5111de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.494+0000 7f81db042740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5105dac0 con 0x557d510f65e0 2023-11-24T06:17:24.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.494+0000 7f81db042740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d50ffa2f0 con 0x557d510f1ba0 2023-11-24T06:17:24.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.494+0000 7f81db042740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d51017c70 con 0x557d510f6ea0 2023-11-24T06:17:24.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.494+0000 7f81db042740 10 client.0 _ll_register_callbacks cb 0x557d5101c3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:17:24.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.494+0000 7f81d5a48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f65e0 0x557d5111fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.494+0000 7f81d5a48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f65e0 0x557d5111fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37474/0 (socket says 172.21.15.5:37474) 2023-11-24T06:17:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.494+0000 7f81d5a48640 1 -- 172.21.15.5:0/2167198467 learned_addr learned my addr 172.21.15.5:0/2167198467 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.495+0000 7f81d6249640 1 --2- 172.21.15.5:0/2167198467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 0x557d5111a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.495+0000 7f81d6a4a640 1 --2- 172.21.15.5:0/2167198467 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f6ea0 0x557d5111de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.495+0000 7f81d5a48640 1 -- 172.21.15.5:0/2167198467 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f6ea0 msgr2=0x557d5111de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.495+0000 7f81d5a48640 1 --2- 172.21.15.5:0/2167198467 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f6ea0 0x557d5111de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.495+0000 7f81d5a48640 1 -- 172.21.15.5:0/2167198467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 msgr2=0x557d5111a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.495+0000 7f81d5a48640 1 --2- 172.21.15.5:0/2167198467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 0x557d5111a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.495+0000 7f81d5a48640 1 -- 172.21.15.5:0/2167198467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d00ca3d0 con 0x557d510f65e0 2023-11-24T06:17:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.495+0000 7f81d5a48640 1 --2- 172.21.15.5:0/2167198467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f65e0 0x557d5111fcb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f81c0005aa0 tx=0x7f81d00d0bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f74efcf4e86df2d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.495+0000 7f81b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-11-24T06:17:24.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.495+0000 7f81b17fa640 1 -- 172.21.15.5:0/2167198467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c000c050 con 0x557d510f65e0 2023-11-24T06:17:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.495+0000 7f81b17fa640 1 -- 172.21.15.5:0/2167198467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81c000ee50 con 0x557d510f65e0 2023-11-24T06:17:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.495+0000 7f81db042740 1 -- 172.21.15.5:0/2167198467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d51056aa0 con 0x557d510f65e0 2023-11-24T06:17:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.496+0000 7f81b17fa640 1 -- 172.21.15.5:0/2167198467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81c0013890 con 0x557d510f65e0 2023-11-24T06:17:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.496+0000 7f81db042740 10 client.5376 fetch_fsmap learned FSMap version 17 2023-11-24T06:17:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.496+0000 7f81db042740 1 -- 172.21.15.5:0/2167198467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x557d5105c140 con 0x557d510f65e0 2023-11-24T06:17:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.496+0000 7f81b17fa640 1 -- 172.21.15.5:0/2167198467 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557d51056aa0 con 0x557d510f65e0 2023-11-24T06:17:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.496+0000 7f81b17fa640 1 -- 172.21.15.5:0/2167198467 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f81c00139f0 con 0x557d510f65e0 2023-11-24T06:17:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.496+0000 7f81db042740 10 client.5376 fetch_fsmap finished waiting for FSMap version 17 2023-11-24T06:17:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.496+0000 7f81db042740 10 client.5376 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T06:17:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.496+0000 7f81db042740 20 client.5376 populate_metadata read hostname 'smithi005' 2023-11-24T06:17:24.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.496+0000 7f81db042740 1 --2- 172.21.15.5:0/2167198467 >> [v2:172.21.15.5:6835/3690265514,v1:172.21.15.5:6837/3690265514] conn(0x557d51141bc0 0x557d51142d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.496+0000 7f81db042740 4 client.5376 mds_command: new command op to 5217 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T06:17:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.496+0000 7f81db042740 1 -- 172.21.15.5:0/2167198467 --> [v2:172.21.15.5:6835/3690265514,v1:172.21.15.5:6837/3690265514] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d510567c0 con 0x557d51141bc0 2023-11-24T06:17:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.497+0000 7f81d6249640 1 --2- 172.21.15.5:0/2167198467 >> [v2:172.21.15.5:6835/3690265514,v1:172.21.15.5:6837/3690265514] conn(0x557d51141bc0 0x557d51142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.497+0000 7f81d6249640 1 --2- 172.21.15.5:0/2167198467 >> [v2:172.21.15.5:6835/3690265514,v1:172.21.15.5:6837/3690265514] conn(0x557d51141bc0 0x557d51142d60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9826b8fdcc199b2 server_cookie=ef5f00ba4ed96d15 in_seq=0 out_seq=0 2023-11-24T06:17:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.497+0000 7f81b17fa640 10 client.5376 ms_handle_connect on v2:172.21.15.5:6835/3690265514 2023-11-24T06:17:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.498+0000 7f81b17fa640 1 -- 172.21.15.5:0/2167198467 <== mds.0 v2:172.21.15.5:6835/3690265514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d510567c0 con 0x557d51141bc0 2023-11-24T06:17:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.498+0000 7f81b17fa640 10 client.5376 handle_command_reply: tid=0 2023-11-24T06:17:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.498+0000 7f81db042740 1 client.5376 shutdown 2023-11-24T06:17:24.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.498+0000 7f81db042740 1 -- 172.21.15.5:0/2167198467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f65e0 msgr2=0x557d5111fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.498+0000 7f81db042740 1 --2- 172.21.15.5:0/2167198467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f65e0 0x557d5111fcb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f81c0005aa0 tx=0x7f81d00d0bc0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.498+0000 7f81db042740 1 -- 172.21.15.5:0/2167198467 shutdown_connections 2023-11-24T06:17:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.498+0000 7f81db042740 1 --2- 172.21.15.5:0/2167198467 >> [v2:172.21.15.5:6835/3690265514,v1:172.21.15.5:6837/3690265514] conn(0x557d51141bc0 0x557d51142d60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.498+0000 7f81db042740 1 --2- 172.21.15.5:0/2167198467 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f6ea0 0x557d5111de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.498+0000 7f81db042740 1 --2- 172.21.15.5:0/2167198467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f65e0 0x557d5111fcb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.498+0000 7f81db042740 1 --2- 172.21.15.5:0/2167198467 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 0x557d5111a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.498+0000 7f81b17fa640 0 client.5376 ms_handle_reset on v2:172.21.15.5:6835/3690265514 2023-11-24T06:17:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.498+0000 7f81db042740 1 -- 172.21.15.5:0/2167198467 >> 172.21.15.5:0/2167198467 conn(0x557d510b8c70 msgr2=0x557d5113c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.499+0000 7f81db042740 1 -- 172.21.15.5:0/2167198467 shutdown_connections 2023-11-24T06:17:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.499+0000 7f81db042740 1 -- 172.21.15.5:0/2167198467 wait complete. 2023-11-24T06:17:24.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.499+0000 7f81db042740 20 client.5376 trim_cache size 0 max 16384 2023-11-24T06:17:24.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.516+0000 7f81db042740 1 Processor -- start 2023-11-24T06:17:24.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.516+0000 7f81db042740 1 -- start start 2023-11-24T06:17:24.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81db042740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 0x557d5111d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81db042740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f65e0 0x557d5111c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81db042740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f6ea0 0x557d5111cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81db042740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d50dc0f20 con 0x557d510f6ea0 2023-11-24T06:17:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81db042740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d5111f460 con 0x557d510f1ba0 2023-11-24T06:17:24.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81db042740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d510f77a0 con 0x557d510f65e0 2023-11-24T06:17:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81d6a4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f6ea0 0x557d5111cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81d6a4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f6ea0 0x557d5111cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37490/0 (socket says 172.21.15.5:37490) 2023-11-24T06:17:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81d6a4a640 1 -- 172.21.15.5:0/1564950488 learned_addr learned my addr 172.21.15.5:0/1564950488 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81d6249640 1 --2- 172.21.15.5:0/1564950488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 0x557d5111d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81d5a48640 1 --2- 172.21.15.5:0/1564950488 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f65e0 0x557d5111c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81d6a4a640 1 -- 172.21.15.5:0/1564950488 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f65e0 msgr2=0x557d5111c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81d6a4a640 1 --2- 172.21.15.5:0/1564950488 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f65e0 0x557d5111c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81d6a4a640 1 -- 172.21.15.5:0/1564950488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 msgr2=0x557d5111d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81d6a4a640 1 --2- 172.21.15.5:0/1564950488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 0x557d5111d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.517+0000 7f81d6a4a640 1 -- 172.21.15.5:0/1564950488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d50ff9b80 con 0x557d510f6ea0 2023-11-24T06:17:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.518+0000 7f81d6a4a640 1 --2- 172.21.15.5:0/1564950488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f6ea0 0x557d5111cce0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f81c802c060 tx=0x7f81c8001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9746cfc0f95658 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.518+0000 7f81c4ff9640 1 -- 172.21.15.5:0/1564950488 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c8003b70 con 0x557d510f6ea0 2023-11-24T06:17:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.518+0000 7f81c4ff9640 1 -- 172.21.15.5:0/1564950488 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c8006b50 con 0x557d510f6ea0 2023-11-24T06:17:24.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.518+0000 7f81c4ff9640 1 -- 172.21.15.5:0/1564950488 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c80050d0 con 0x557d510f6ea0 2023-11-24T06:17:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.519+0000 7f81db042740 1 -- 172.21.15.5:0/1564950488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f6ea0 msgr2=0x557d5111cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.519+0000 7f81db042740 1 --2- 172.21.15.5:0/1564950488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f6ea0 0x557d5111cce0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f81c802c060 tx=0x7f81c8001510 comp rx=0 tx=0).stop 2023-11-24T06:17:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.519+0000 7f81db042740 1 -- 172.21.15.5:0/1564950488 shutdown_connections 2023-11-24T06:17:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.519+0000 7f81db042740 1 --2- 172.21.15.5:0/1564950488 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f65e0 0x557d5111c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.519+0000 7f81db042740 1 --2- 172.21.15.5:0/1564950488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f6ea0 0x557d5111cce0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.519+0000 7f81db042740 1 --2- 172.21.15.5:0/1564950488 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f1ba0 0x557d5111d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.519+0000 7f81db042740 1 -- 172.21.15.5:0/1564950488 >> 172.21.15.5:0/1564950488 conn(0x557d510b8c70 msgr2=0x557d51120d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.519+0000 7f81db042740 1 -- 172.21.15.5:0/1564950488 shutdown_connections 2023-11-24T06:17:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.519+0000 7f81db042740 1 -- 172.21.15.5:0/1564950488 wait complete. 2023-11-24T06:17:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.519+0000 7f81db042740 1 -- start start 2023-11-24T06:17:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.520+0000 7f81db042740 1 Processor -- start 2023-11-24T06:17:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81db042740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f1ba0 0x557d5114b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81db042740 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f65e0 0x557d511a6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81db042740 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f6ea0 0x557d511aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81db042740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d511a6b60 con 0x557d510f1ba0 2023-11-24T06:17:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81db042740 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d511aaec0 con 0x557d510f65e0 2023-11-24T06:17:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81db042740 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d510f14b0 con 0x557d510f6ea0 2023-11-24T06:17:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81db042740 10 client.0 _ll_register_callbacks cb 0x7f81c0001690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T06:17:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81d6249640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f1ba0 0x557d5114b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81d6249640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f1ba0 0x557d5114b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37494/0 (socket says 172.21.15.5:37494) 2023-11-24T06:17:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81d6249640 1 -- 172.21.15.5:0/415116840 learned_addr learned my addr 172.21.15.5:0/415116840 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81d6249640 1 -- 172.21.15.5:0/415116840 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f6ea0 msgr2=0x557d511aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81d6249640 1 --2- 172.21.15.5:0/415116840 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f6ea0 0x557d511aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81d6249640 1 -- 172.21.15.5:0/415116840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f65e0 msgr2=0x557d511a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81d6249640 1 --2- 172.21.15.5:0/415116840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f65e0 0x557d511a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.521+0000 7f81d6249640 1 -- 172.21.15.5:0/415116840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc003520 con 0x557d510f1ba0 2023-11-24T06:17:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.522+0000 7f81d6249640 1 --2- 172.21.15.5:0/415116840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f1ba0 0x557d5114b4b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f81cc058440 tx=0x7f81cc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a43fe93a2f4bcc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.522+0000 7f81b17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-11-24T06:17:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.522+0000 7f81b17fa640 1 -- 172.21.15.5:0/415116840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc058890 con 0x557d510f1ba0 2023-11-24T06:17:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.522+0000 7f81b17fa640 1 -- 172.21.15.5:0/415116840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81cc00ce10 con 0x557d510f1ba0 2023-11-24T06:17:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.522+0000 7f81db042740 1 -- 172.21.15.5:0/415116840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f81cc060dc0 con 0x557d510f1ba0 2023-11-24T06:17:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.522+0000 7f81b17fa640 1 -- 172.21.15.5:0/415116840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81cc05cc10 con 0x557d510f1ba0 2023-11-24T06:17:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.522+0000 7f81db042740 10 client.5382 fetch_fsmap learned FSMap version 17 2023-11-24T06:17:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.522+0000 7f81db042740 1 -- 172.21.15.5:0/415116840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x557d5101f170 con 0x557d510f1ba0 2023-11-24T06:17:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.522+0000 7f81b17fa640 1 -- 172.21.15.5:0/415116840 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc060dc0 con 0x557d510f1ba0 2023-11-24T06:17:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.522+0000 7f81b17fa640 1 -- 172.21.15.5:0/415116840 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f81cc0551d0 con 0x557d510f1ba0 2023-11-24T06:17:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.522+0000 7f81db042740 10 client.5382 fetch_fsmap finished waiting for FSMap version 17 2023-11-24T06:17:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.523+0000 7f81db042740 10 client.5382 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T06:17:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.523+0000 7f81db042740 20 client.5382 populate_metadata read hostname 'smithi005' 2023-11-24T06:17:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.523+0000 7f81db042740 1 --2- 172.21.15.5:0/415116840 >> [v2:172.21.15.5:6835/3690265514,v1:172.21.15.5:6837/3690265514] conn(0x557d5111c7a0 0x557d5111cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.523+0000 7f81db042740 4 client.5382 mds_command: new command op to 5217 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T06:17:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.523+0000 7f81db042740 1 -- 172.21.15.5:0/415116840 --> [v2:172.21.15.5:6835/3690265514,v1:172.21.15.5:6837/3690265514] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557d50d5e6c0 con 0x557d5111c7a0 2023-11-24T06:17:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.523+0000 7f81d5a48640 1 --2- 172.21.15.5:0/415116840 >> [v2:172.21.15.5:6835/3690265514,v1:172.21.15.5:6837/3690265514] conn(0x557d5111c7a0 0x557d5111cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.523+0000 7f81b17fa640 10 client.5382 ms_handle_connect on v2:172.21.15.5:6835/3690265514 2023-11-24T06:17:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.523+0000 7f81d5a48640 1 --2- 172.21.15.5:0/415116840 >> [v2:172.21.15.5:6835/3690265514,v1:172.21.15.5:6837/3690265514] conn(0x557d5111c7a0 0x557d5111cb80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=277b9ca02ae1d20c server_cookie=ba27605c75b3fc60 in_seq=0 out_seq=0 2023-11-24T06:17:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81b17fa640 1 -- 172.21.15.5:0/415116840 <== mds.0 v2:172.21.15.5:6835/3690265514 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x557d50d5e6c0 con 0x557d5111c7a0 2023-11-24T06:17:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81b17fa640 10 client.5382 handle_command_reply: tid=0 2023-11-24T06:17:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81db042740 1 client.5382 shutdown 2023-11-24T06:17:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81db042740 1 -- 172.21.15.5:0/415116840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f1ba0 msgr2=0x557d5114b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81db042740 1 --2- 172.21.15.5:0/415116840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f1ba0 0x557d5114b4b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f81cc058440 tx=0x7f81cc000bc0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81db042740 1 -- 172.21.15.5:0/415116840 shutdown_connections 2023-11-24T06:17:24.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81db042740 1 --2- 172.21.15.5:0/415116840 >> [v2:172.21.15.5:6835/3690265514,v1:172.21.15.5:6837/3690265514] conn(0x557d5111c7a0 0x557d5111cb80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81db042740 1 --2- 172.21.15.5:0/415116840 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x557d510f6ea0 0x557d511aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81db042740 1 --2- 172.21.15.5:0/415116840 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x557d510f65e0 0x557d511a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81db042740 1 --2- 172.21.15.5:0/415116840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d510f1ba0 0x557d5114b4b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81b17fa640 0 client.5382 ms_handle_reset on v2:172.21.15.5:6835/3690265514 2023-11-24T06:17:24.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81db042740 1 -- 172.21.15.5:0/415116840 >> 172.21.15.5:0/415116840 conn(0x557d510b8c70 msgr2=0x557d510ed2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:24.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81db042740 1 -- 172.21.15.5:0/415116840 shutdown_connections 2023-11-24T06:17:24.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81db042740 1 -- 172.21.15.5:0/415116840 wait complete. 2023-11-24T06:17:24.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.524+0000 7f81db042740 20 client.5382 trim_cache size 0 max 16384 2023-11-24T06:17:24.532 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-11-24T06:17:24.532 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c", 2023-11-24T06:17:24.532 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2023-11-24T06:17:24.532 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 5217, 2023-11-24T06:17:24.532 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-11-24T06:17:24.532 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-11-24T06:17:24.532 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-11-24T06:17:24.532 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 6.7123527359999997, 2023-11-24T06:17:24.532 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 17, 2023-11-24T06:17:24.532 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 35, 2023-11-24T06:17:24.533 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 35, 2023-11-24T06:17:24.533 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 15.906859339 2023-11-24T06:17:24.533 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-11-24T06:17:24.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.525+0000 7f81d84ad640 1 -- 172.21.15.5:0/1599019295 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f81b406ae70 msgr2=0x7f81b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.525+0000 7f81d84ad640 1 --2- 172.21.15.5:0/1599019295 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f81b406ae70 0x7f81b406d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f81c0001900 tx=0x7f81c0009040 comp rx=0 tx=0).stop 2023-11-24T06:17:24.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.525+0000 7f81d84ad640 1 -- 172.21.15.5:0/1599019295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc005180 msgr2=0x7f81d0151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.525+0000 7f81d84ad640 1 --2- 172.21.15.5:0/1599019295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc005180 0x7f81d0151330 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f81cc010860 tx=0x7f81cc010d20 comp rx=0 tx=0).stop 2023-11-24T06:17:24.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.525+0000 7f81d84ad640 1 -- 172.21.15.5:0/1599019295 shutdown_connections 2023-11-24T06:17:24.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.525+0000 7f81d84ad640 1 --2- 172.21.15.5:0/1599019295 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f81b406ae70 0x7f81b406d330 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.525+0000 7f81d84ad640 1 --2- 172.21.15.5:0/1599019295 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f81d01f6450 0x7f81d01f68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.525+0000 7f81d84ad640 1 --2- 172.21.15.5:0/1599019295 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f81d00cca90 0x7f81d00ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.525+0000 7f81d84ad640 1 --2- 172.21.15.5:0/1599019295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81bc005180 0x7f81d0151330 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.525+0000 7f81d84ad640 1 -- 172.21.15.5:0/1599019295 >> 172.21.15.5:0/1599019295 conn(0x7f81d0071f90 msgr2=0x7f81d00cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:24.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.525+0000 7f81d84ad640 1 -- 172.21.15.5:0/1599019295 shutdown_connections 2023-11-24T06:17:24.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.525+0000 7f81d84ad640 1 -- 172.21.15.5:0/1599019295 wait complete. 2023-11-24T06:17:24.543 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:17:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.747+0000 7fdecf2d1640 1 Processor -- start 2023-11-24T06:17:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.747+0000 7fdecf2d1640 1 -- start start 2023-11-24T06:17:24.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.748+0000 7fdecf2d1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdec813dfd0 0x7fdec815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.748+0000 7fdecf2d1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdec80cf500 0x7fdec806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.748+0000 7fdecf2d1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec80ce7c0 con 0x7fdec80cf050 2023-11-24T06:17:24.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.748+0000 7fdecf2d1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec8140900 con 0x7fdec813dfd0 2023-11-24T06:17:24.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.748+0000 7fdecf2d1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec815fa70 con 0x7fdec80cf500 2023-11-24T06:17:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.749+0000 7fdecead0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdec80cf500 0x7fdec806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.749+0000 7fdece2cf640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdec813dfd0 0x7fdec815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.749+0000 7fdece2cf640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdec813dfd0 0x7fdec815be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:42402/0 (socket says 172.21.15.5:42402) 2023-11-24T06:17:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.749+0000 7fdece2cf640 1 -- 172.21.15.5:0/994932298 learned_addr learned my addr 172.21.15.5:0/994932298 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.749+0000 7fdecead0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdec80cf500 0x7fdec806c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:45314/0 (socket says 172.21.15.5:45314) 2023-11-24T06:17:24.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.750+0000 7fdece2cf640 1 -- 172.21.15.5:0/994932298 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdec80cf500 msgr2=0x7fdec806c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:24.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.750+0000 7fdece2cf640 1 --2- 172.21.15.5:0/994932298 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdec80cf500 0x7fdec806c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.750+0000 7fdece2cf640 1 -- 172.21.15.5:0/994932298 >> v1:172.21.15.6:6789/0 conn(0x7fdec80cf050 legacy=0x7fdec806c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.750+0000 7fdece2cf640 1 -- 172.21.15.5:0/994932298 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec80cf950 con 0x7fdec813dfd0 2023-11-24T06:17:24.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.750+0000 7fdecead0640 1 --2- 172.21.15.5:0/994932298 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdec80cf500 0x7fdec806c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:17:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.751+0000 7fdece2cf640 1 --2- 172.21.15.5:0/994932298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdec813dfd0 0x7fdec815be70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009870 tx=0x7fdec402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fec0ea2cdb9c8f75 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.752+0000 7fdecd2cd640 1 -- 172.21.15.5:0/994932298 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4030840 con 0x7fdec813dfd0 2023-11-24T06:17:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.752+0000 7fdecd2cd640 1 -- 172.21.15.5:0/994932298 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec40309a0 con 0x7fdec813dfd0 2023-11-24T06:17:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.752+0000 7fdecd2cd640 1 -- 172.21.15.5:0/994932298 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec4030d30 con 0x7fdec813dfd0 2023-11-24T06:17:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.752+0000 7fdecf2d1640 1 -- 172.21.15.5:0/994932298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdec813dfd0 msgr2=0x7fdec815be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.752+0000 7fdecf2d1640 1 --2- 172.21.15.5:0/994932298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdec813dfd0 0x7fdec815be70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdec4009870 tx=0x7fdec402b980 comp rx=0 tx=0).stop 2023-11-24T06:17:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.752+0000 7fdecf2d1640 1 -- 172.21.15.5:0/994932298 shutdown_connections 2023-11-24T06:17:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.752+0000 7fdecf2d1640 1 --2- 172.21.15.5:0/994932298 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdec80cf500 0x7fdec806c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.752+0000 7fdecf2d1640 1 --2- 172.21.15.5:0/994932298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdec813dfd0 0x7fdec815be70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.752+0000 7fdecf2d1640 1 -- 172.21.15.5:0/994932298 >> 172.21.15.5:0/994932298 conn(0x7fdec806a860 msgr2=0x7fdec806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.752+0000 7fdecf2d1640 1 -- 172.21.15.5:0/994932298 shutdown_connections 2023-11-24T06:17:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.752+0000 7fdecf2d1640 1 -- 172.21.15.5:0/994932298 wait complete. 2023-11-24T06:17:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.754+0000 7fdecf2d1640 1 Processor -- start 2023-11-24T06:17:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.754+0000 7fdecf2d1640 1 -- start start 2023-11-24T06:17:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.755+0000 7fdecf2d1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdec80cf050 0x7fdec8156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.755+0000 7fdecf2d1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdec80cf500 0x7fdec8159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.755+0000 7fdecf2d1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec813dfd0 0x7fdec8159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.755+0000 7fdecf2d1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec814dab0 con 0x7fdec813dfd0 2023-11-24T06:17:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.755+0000 7fdecf2d1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec80b1a30 con 0x7fdec80cf500 2023-11-24T06:17:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.755+0000 7fdecf2d1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec81576a0 con 0x7fdec80cf050 2023-11-24T06:17:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.756+0000 7fdecead0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec813dfd0 0x7fdec8159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.756+0000 7fdecead0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec813dfd0 0x7fdec8159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37500/0 (socket says 172.21.15.5:37500) 2023-11-24T06:17:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.756+0000 7fdecead0640 1 -- 172.21.15.5:0/3800327992 learned_addr learned my addr 172.21.15.5:0/3800327992 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.756+0000 7fdece2cf640 1 --2- 172.21.15.5:0/3800327992 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdec80cf050 0x7fdec8156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.756+0000 7fdecdace640 1 --2- 172.21.15.5:0/3800327992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdec80cf500 0x7fdec8159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.756+0000 7fdecead0640 1 -- 172.21.15.5:0/3800327992 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdec80cf050 msgr2=0x7fdec8156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.756+0000 7fdecead0640 1 --2- 172.21.15.5:0/3800327992 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdec80cf050 0x7fdec8156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.756+0000 7fdecead0640 1 -- 172.21.15.5:0/3800327992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdec80cf500 msgr2=0x7fdec8159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.756+0000 7fdecead0640 1 --2- 172.21.15.5:0/3800327992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdec80cf500 0x7fdec8159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.756+0000 7fdecead0640 1 -- 172.21.15.5:0/3800327992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec0009a70 con 0x7fdec813dfd0 2023-11-24T06:17:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.756+0000 7fdecead0640 1 --2- 172.21.15.5:0/3800327992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec813dfd0 0x7fdec8159640 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdec000efc0 tx=0x7fdec0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.757+0000 7fdebeffd640 1 -- 172.21.15.5:0/3800327992 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0009d70 con 0x7fdec813dfd0 2023-11-24T06:17:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.757+0000 7fdebeffd640 1 -- 172.21.15.5:0/3800327992 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec000f040 con 0x7fdec813dfd0 2023-11-24T06:17:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.757+0000 7fdebeffd640 1 -- 172.21.15.5:0/3800327992 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0003990 con 0x7fdec813dfd0 2023-11-24T06:17:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.757+0000 7fdecf2d1640 1 -- 172.21.15.5:0/3800327992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec40094d0 con 0x7fdec813dfd0 2023-11-24T06:17:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.757+0000 7fdecf2d1640 1 -- 172.21.15.5:0/3800327992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec80b1fb0 con 0x7fdec813dfd0 2023-11-24T06:17:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.758+0000 7fdecf2d1640 1 -- 172.21.15.5:0/3800327992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec804e840 con 0x7fdec813dfd0 2023-11-24T06:17:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.759+0000 7fdebeffd640 1 -- 172.21.15.5:0/3800327992 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdec00144a0 con 0x7fdec813dfd0 2023-11-24T06:17:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.760+0000 7fdebeffd640 1 --2- 172.21.15.5:0/3800327992 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdeac06ae70 0x7fdeac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.760+0000 7fdebeffd640 1 -- 172.21.15.5:0/3800327992 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdec008a290 con 0x7fdec813dfd0 2023-11-24T06:17:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.760+0000 7fdece2cf640 1 --2- 172.21.15.5:0/3800327992 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdeac06ae70 0x7fdeac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.761+0000 7fdece2cf640 1 --2- 172.21.15.5:0/3800327992 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdeac06ae70 0x7fdeac06d330 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdec4036000 tx=0x7fdec4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:24.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.765+0000 7fdebeffd640 1 -- 172.21.15.5:0/3800327992 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdec0055330 con 0x7fdec813dfd0 2023-11-24T06:17:24.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.987+0000 7fdecf2d1640 1 -- 172.21.15.5:0/3800327992 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdec80ce9a0 con 0x7fdec813dfd0 2023-11-24T06:17:24.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.988+0000 7fdebeffd640 1 -- 172.21.15.5:0/3800327992 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdec0058950 con 0x7fdec813dfd0 2023-11-24T06:17:24.991 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:17:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.990+0000 7fdecf2d1640 1 -- 172.21.15.5:0/3800327992 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdeac06ae70 msgr2=0x7fdeac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.990+0000 7fdecf2d1640 1 --2- 172.21.15.5:0/3800327992 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdeac06ae70 0x7fdeac06d330 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdec4036000 tx=0x7fdec4008030 comp rx=0 tx=0).stop 2023-11-24T06:17:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.990+0000 7fdecf2d1640 1 -- 172.21.15.5:0/3800327992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec813dfd0 msgr2=0x7fdec8159640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:24.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.990+0000 7fdecf2d1640 1 --2- 172.21.15.5:0/3800327992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec813dfd0 0x7fdec8159640 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fdec000efc0 tx=0x7fdec0009f90 comp rx=0 tx=0).stop 2023-11-24T06:17:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.990+0000 7fdecf2d1640 1 -- 172.21.15.5:0/3800327992 shutdown_connections 2023-11-24T06:17:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.990+0000 7fdecf2d1640 1 --2- 172.21.15.5:0/3800327992 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdeac06ae70 0x7fdeac06d330 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.990+0000 7fdecf2d1640 1 --2- 172.21.15.5:0/3800327992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdec813dfd0 0x7fdec8159640 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.990+0000 7fdecf2d1640 1 --2- 172.21.15.5:0/3800327992 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdec80cf500 0x7fdec8159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.990+0000 7fdecf2d1640 1 --2- 172.21.15.5:0/3800327992 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdec80cf050 0x7fdec8156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:24.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.991+0000 7fdecf2d1640 1 -- 172.21.15.5:0/3800327992 >> 172.21.15.5:0/3800327992 conn(0x7fdec806a860 msgr2=0x7fdec8159e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:24.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.991+0000 7fdecf2d1640 1 -- 172.21.15.5:0/3800327992 shutdown_connections 2023-11-24T06:17:24.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:24.991+0000 7fdecf2d1640 1 -- 172.21.15.5:0/3800327992 wait complete. 2023-11-24T06:17:25.010 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:17:25.011 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:17:25.011 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2023-11-24T06:17:25.011 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T06:17:25.011 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T06:17:25.011 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:17:25.011 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:17:25.012 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:17:25.012 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:17:25.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:25.012 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2023-11-24T06:17:25.065 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:17:25.065 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:17:25.065 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:17:25.065 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:17:25.065 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2023-11-24T06:17:25.065 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:17:25.065 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:17:25.066 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2023-11-24T06:17:25.066 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2023-11-24T06:17:25.066 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:17:25.066 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2023-11-24T06:17:25.066 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2023-11-24T06:17:25.066 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:17:25.066 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:17:25.067 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2023-11-24T06:17:25.067 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T06:17:25.067 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42247sec preferred_lft 42247sec 2023-11-24T06:17:25.067 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link noprefixroute 2023-11-24T06:17:25.067 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:17:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:17:25.067 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:17:25.067 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2023-11-24T06:17:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:17:25.067 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether ea:40:59:30:3f:75 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:17:25.067 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:17:25.068 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:17:25.068 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::a4c7:50ff:feb3:591d/64 scope link 2023-11-24T06:17:25.068 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:17:25.068 INFO:teuthology.orchestra.run.smithi005.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:17:25.068 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether f6:90:1f:88:d8:74 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:17:25.068 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::f490:1fff:fe88:d874/64 scope link 2023-11-24T06:17:25.068 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:17:25.068 INFO:teuthology.orchestra.run.smithi005.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:17:25.068 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether ea:40:59:30:3f:75 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:17:25.068 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::e840:59ff:fe30:3f75/64 scope link 2023-11-24T06:17:25.068 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:17:25.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:25.069 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:17:25.169 INFO:teuthology.orchestra.run.smithi005.stdout:1 2023-11-24T06:17:25.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:25.182 DEBUG:teuthology.orchestra.run.smithi005:> route 2023-11-24T06:17:25.241 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2023-11-24T06:17:25.242 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:17:25.242 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T06:17:25.242 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T06:17:25.242 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:17:25.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:25.243 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-11-24T06:17:25.244 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-11-24T06:17:25.244 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T06:17:25.244 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T06:17:25.244 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T06:17:25.244 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-11-24T06:17:25.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:25.450 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2023-11-24T06:17:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:17:25.507 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:17:25.508 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:17:25.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:25.509 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2023-11-24T06:17:25.603 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-24T06:17:25.604 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:17:25.604 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T06:17:25.658 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T06:17:25.659 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:17:25.659 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:17:25.716 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2023-11-24T06:17:25.820 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:17:25.883 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2023-11-24T06:17:25.883 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2023-11-24T06:17:25.883 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2023-11-24T06:17:25.883 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2023-11-24T06:17:25.883 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2023-11-24T06:17:25.883 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2023-11-24T06:17:25.883 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2023-11-24T06:17:25.883 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2023-11-24T06:17:25.883 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2023-11-24T06:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2023-11-24T06:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2023-11-24T06:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2023-11-24T06:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2023-11-24T06:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2023-11-24T06:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2023-11-24T06:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2023-11-24T06:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2023-11-24T06:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:17:25.884 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2023-11-24T06:17:25.885 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2023-11-24T06:17:25.885 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:17:25.885 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:17:25.885 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:17:25.885 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:17:25.885 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2023-11-24T06:17:25.885 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:17:25.885 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:17:25.885 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:17:25.885 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:17:25.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:25.886 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2023-11-24T06:17:25.944 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:17:25.944 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-24T06:17:25.950 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2023-11-24T06:17:26.035 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:17:26.084 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2023-11-24T06:17:26.084 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2023-11-24T06:17:26.084 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2023-11-24T06:17:26.084 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2023-11-24T06:17:26.085 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2023-11-24T06:17:26.085 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2023-11-24T06:17:26.085 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2023-11-24T06:17:26.085 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2023-11-24T06:17:26.085 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2023-11-24T06:17:26.085 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2023-11-24T06:17:26.086 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2023-11-24T06:17:26.086 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:17:26.086 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2023-11-24T06:17:26.086 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2023-11-24T06:17:26.087 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2023-11-24T06:17:26.087 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2023-11-24T06:17:26.088 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2023-11-24T06:17:26.088 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2023-11-24T06:17:26.088 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:17:26.088 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:17:26.088 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:17:26.089 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:17:26.089 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2023-11-24T06:17:26.089 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2023-11-24T06:17:26.089 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2023-11-24T06:17:26.089 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2023-11-24T06:17:26.090 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2023-11-24T06:17:26.090 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2023-11-24T06:17:26.090 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2023-11-24T06:17:26.090 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:17:26.090 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:17:26.091 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:17:26.091 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:17:26.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:26.092 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2023-11-24T06:17:26.145 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:2023-11-24T06:17:26.143+0000 7f7a79840480 -1 init, newargv = 0x5605612aee30 newargc=15 2023-11-24T06:17:26.146 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[101461]: starting ceph client 2023-11-24T06:17:26.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[101461]: starting fuse 2023-11-24T06:17:26.178 INFO:teuthology.orchestra.run.smithi005.stdout:40 2023-11-24T06:17:26.179 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-24T06:17:26.179 DEBUG:teuthology.orchestra.run.smithi005:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:17:26.179 DEBUG:teuthology.orchestra.run.smithi005:> import glob 2023-11-24T06:17:26.179 DEBUG:teuthology.orchestra.run.smithi005:> import re 2023-11-24T06:17:26.179 DEBUG:teuthology.orchestra.run.smithi005:> import os 2023-11-24T06:17:26.180 DEBUG:teuthology.orchestra.run.smithi005:> import subprocess 2023-11-24T06:17:26.180 DEBUG:teuthology.orchestra.run.smithi005:> 2023-11-24T06:17:26.180 DEBUG:teuthology.orchestra.run.smithi005:> def _find_admin_socket(client_name): 2023-11-24T06:17:26.180 DEBUG:teuthology.orchestra.run.smithi005:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-24T06:17:26.180 DEBUG:teuthology.orchestra.run.smithi005:> files = glob.glob(asok_path) 2023-11-24T06:17:26.180 DEBUG:teuthology.orchestra.run.smithi005:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-24T06:17:26.180 DEBUG:teuthology.orchestra.run.smithi005:> 2023-11-24T06:17:26.181 DEBUG:teuthology.orchestra.run.smithi005:> # Given a non-glob path, it better be there 2023-11-24T06:17:26.181 DEBUG:teuthology.orchestra.run.smithi005:> if "*" not in asok_path: 2023-11-24T06:17:26.181 DEBUG:teuthology.orchestra.run.smithi005:> assert(len(files) == 1) 2023-11-24T06:17:26.181 DEBUG:teuthology.orchestra.run.smithi005:> return files[0] 2023-11-24T06:17:26.181 DEBUG:teuthology.orchestra.run.smithi005:> 2023-11-24T06:17:26.181 DEBUG:teuthology.orchestra.run.smithi005:> for f in files: 2023-11-24T06:17:26.181 DEBUG:teuthology.orchestra.run.smithi005:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:17:26.182 DEBUG:teuthology.orchestra.run.smithi005:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:17:26.182 DEBUG:teuthology.orchestra.run.smithi005:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:17:26.182 DEBUG:teuthology.orchestra.run.smithi005:> contents = proc_f.read() 2023-11-24T06:17:26.182 DEBUG:teuthology.orchestra.run.smithi005:> if mountpoint in contents: 2023-11-24T06:17:26.182 DEBUG:teuthology.orchestra.run.smithi005:> return f 2023-11-24T06:17:26.182 DEBUG:teuthology.orchestra.run.smithi005:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:17:26.182 DEBUG:teuthology.orchestra.run.smithi005:> 2023-11-24T06:17:26.183 DEBUG:teuthology.orchestra.run.smithi005:> print(_find_admin_socket("client.0")) 2023-11-24T06:17:26.183 DEBUG:teuthology.orchestra.run.smithi005:> ' 2023-11-24T06:17:26.383 INFO:teuthology.orchestra.run.smithi005.stdout:/var/run/ceph/ceph-client.0.101461.asok 2023-11-24T06:17:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:17:26.402 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.101461.asok 2023-11-24T06:17:26.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:26.402 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.101461.asok status 2023-11-24T06:17:26.653 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-11-24T06:17:26.653 INFO:teuthology.orchestra.run.smithi005.stdout: "metadata": { 2023-11-24T06:17:26.654 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:17:26.654 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:17:26.654 INFO:teuthology.orchestra.run.smithi005.stdout: "entity_id": "0", 2023-11-24T06:17:26.654 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": "smithi005", 2023-11-24T06:17:26.654 INFO:teuthology.orchestra.run.smithi005.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-24T06:17:26.654 INFO:teuthology.orchestra.run.smithi005.stdout: "pid": "101461", 2023-11-24T06:17:26.654 INFO:teuthology.orchestra.run.smithi005.stdout: "root": "/" 2023-11-24T06:17:26.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:26.654 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_count": 0, 2023-11-24T06:17:26.654 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_pinned_count": 0, 2023-11-24T06:17:26.654 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 5394, 2023-11-24T06:17:26.654 INFO:teuthology.orchestra.run.smithi005.stdout: "inst": { 2023-11-24T06:17:26.654 INFO:teuthology.orchestra.run.smithi005.stdout: "name": { 2023-11-24T06:17:26.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "client", 2023-11-24T06:17:26.655 INFO:teuthology.orchestra.run.smithi005.stdout: "num": 5394 2023-11-24T06:17:26.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:26.655 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2023-11-24T06:17:26.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2023-11-24T06:17:26.655 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:17:26.655 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1713937792 2023-11-24T06:17:26.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:26.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:26.655 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2023-11-24T06:17:26.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2023-11-24T06:17:26.655 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:17:26.656 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1713937792 2023-11-24T06:17:26.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-11-24T06:17:26.656 INFO:teuthology.orchestra.run.smithi005.stdout: "inst_str": "client.5394 192.168.0.1:0/1713937792", 2023-11-24T06:17:26.656 INFO:teuthology.orchestra.run.smithi005.stdout: "addr_str": "192.168.0.1:0/1713937792", 2023-11-24T06:17:26.656 INFO:teuthology.orchestra.run.smithi005.stdout: "inode_count": 1, 2023-11-24T06:17:26.656 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_epoch": 17, 2023-11-24T06:17:26.656 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch": 35, 2023-11-24T06:17:26.656 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:17:26.656 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklisted": false, 2023-11-24T06:17:26.656 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs" 2023-11-24T06:17:26.656 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-11-24T06:17:26.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:26.669 DEBUG:teuthology.orchestra.run.smithi005:> stat --file-system '--printf=%T 2023-11-24T06:17:26.669 DEBUG:teuthology.orchestra.run.smithi005:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-24T06:17:26.724 INFO:teuthology.orchestra.run.smithi005.stdout:fuseblk 2023-11-24T06:17:26.724 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-24T06:17:26.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:26.725 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T06:17:26.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T06:17:27.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.055+0000 7f9adffff640 1 Processor -- start 2023-11-24T06:17:27.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.056+0000 7f9adffff640 1 -- start start 2023-11-24T06:17:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.058+0000 7f9adffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ae00cf760 0x7f9ae015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.058+0000 7f9adffff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae0071700 con 0x7f9ae0143950 2023-11-24T06:17:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.058+0000 7f9adffff640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae015b410 con 0x7f9ae0143570 2023-11-24T06:17:27.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.058+0000 7f9adffff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae015b130 con 0x7f9ae00cf760 2023-11-24T06:17:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.058+0000 7f9adf7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f9ae0143950 0x7f9ae00b1b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35886/0 (socket says 172.21.15.5:35886) 2023-11-24T06:17:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.058+0000 7f9adf7fe640 1 -- 172.21.15.5:0/2196970299 learned_addr learned my addr 172.21.15.5:0/2196970299 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.059+0000 7f9ad7fff640 1 --2- 172.21.15.5:0/2196970299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ae00cf760 0x7f9ae015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.059+0000 7f9ade7fc640 1 -- 172.21.15.5:0/2196970299 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002065653 0 0) 0x7f9ad8001490 con 0x7f9ae0143950 2023-11-24T06:17:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.059+0000 7f9ade7fc640 1 -- 172.21.15.5:0/2196970299 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9acc003520 con 0x7f9ae0143950 2023-11-24T06:17:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.059+0000 7f9ade7fc640 1 -- 172.21.15.5:0/2196970299 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4066965285 0 0) 0x7f9acc003520 con 0x7f9ae0143950 2023-11-24T06:17:27.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.059+0000 7f9ade7fc640 1 -- 172.21.15.5:0/2196970299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ae00cf760 msgr2=0x7f9ae015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.059+0000 7f9ade7fc640 1 --2- 172.21.15.5:0/2196970299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ae00cf760 0x7f9ae015a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.059+0000 7f9ade7fc640 1 -- 172.21.15.5:0/2196970299 >> v1:172.21.15.6:6789/0 conn(0x7f9ae0143570 legacy=0x7f9ae00cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.059+0000 7f9ade7fc640 1 -- 172.21.15.5:0/2196970299 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae015dbd0 con 0x7f9ae0143950 2023-11-24T06:17:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.059+0000 7f9ade7fc640 1 -- 172.21.15.5:0/2196970299 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f9ad8001b80 con 0x7f9ae0143950 2023-11-24T06:17:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.060+0000 7f9ade7fc640 1 -- 172.21.15.5:0/2196970299 >> v1:172.21.15.5:6789/0 conn(0x7f9ae0143950 legacy=0x7f9ae00b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.060+0000 7f9ade7fc640 1 --2- 172.21.15.5:0/2196970299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9acc003a10 0x7f9acc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.060+0000 7f9ade7fc640 1 --2- 172.21.15.5:0/2196970299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ae00cf760 0x7f9acc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.060+0000 7f9ade7fc640 1 --2- 172.21.15.5:0/2196970299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc004d00 0x7f9acc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.060+0000 7f9ade7fc640 1 -- 172.21.15.5:0/2196970299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae015b130 con 0x7f9acc003a10 2023-11-24T06:17:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.060+0000 7f9ade7fc640 1 -- 172.21.15.5:0/2196970299 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc00ba80 con 0x7f9acc004d00 2023-11-24T06:17:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.060+0000 7f9ade7fc640 1 -- 172.21.15.5:0/2196970299 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc00bc40 con 0x7f9ae00cf760 2023-11-24T06:17:27.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.060+0000 7f9adeffd640 1 --2- 172.21.15.5:0/2196970299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9acc003a10 0x7f9acc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.060+0000 7f9adeffd640 1 -- 172.21.15.5:0/2196970299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ae00cf760 msgr2=0x7f9acc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.060+0000 7f9adeffd640 1 --2- 172.21.15.5:0/2196970299 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ae00cf760 0x7f9acc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.060+0000 7f9adeffd640 1 -- 172.21.15.5:0/2196970299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc004d00 msgr2=0x7f9acc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.060+0000 7f9adeffd640 1 --2- 172.21.15.5:0/2196970299 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9acc004d00 0x7f9acc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.060+0000 7f9adeffd640 1 -- 172.21.15.5:0/2196970299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc003520 con 0x7f9acc003a10 2023-11-24T06:17:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.061+0000 7f9adeffd640 1 --2- 172.21.15.5:0/2196970299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9acc003a10 0x7f9acc003df0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0001860 tx=0x7f9acc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8cfa20cec0e928 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.062+0000 7f9ade7fc640 1 -- 172.21.15.5:0/2196970299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0033630 con 0x7f9acc003a10 2023-11-24T06:17:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.062+0000 7f9ade7fc640 1 -- 172.21.15.5:0/2196970299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad00337b0 con 0x7f9acc003a10 2023-11-24T06:17:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.062+0000 7f9ade7fc640 1 -- 172.21.15.5:0/2196970299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0033b60 con 0x7f9acc003a10 2023-11-24T06:17:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.062+0000 7f9adffff640 1 -- 172.21.15.5:0/2196970299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9acc003a10 msgr2=0x7f9acc003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.062+0000 7f9adffff640 1 --2- 172.21.15.5:0/2196970299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9acc003a10 0x7f9acc003df0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0001860 tx=0x7f9acc0057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.062+0000 7f9adf7fe640 1 -- 172.21.15.5:0/2196970299 reap_dead start 2023-11-24T06:17:27.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.062+0000 7f9adffff640 1 -- 172.21.15.5:0/2196970299 shutdown_connections 2023-11-24T06:17:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.062+0000 7f9adffff640 1 -- 172.21.15.5:0/2196970299 >> 172.21.15.5:0/2196970299 conn(0x7f9ae006a860 msgr2=0x7f9ae006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.062+0000 7f9adffff640 1 -- 172.21.15.5:0/2196970299 shutdown_connections 2023-11-24T06:17:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.062+0000 7f9adffff640 1 -- 172.21.15.5:0/2196970299 wait complete. 2023-11-24T06:17:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.064+0000 7f9adffff640 1 Processor -- start 2023-11-24T06:17:27.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.064+0000 7f9adffff640 1 -- start start 2023-11-24T06:17:27.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.065+0000 7f9adffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9acc003a10 0x7f9ae013a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:27.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.065+0000 7f9adffff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ae0152bf0 0x7f9ae006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:27.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.065+0000 7f9adffff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ae006d840 0x7f9ae0150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:27.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.065+0000 7f9adffff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae006dcc0 con 0x7f9acc003a10 2023-11-24T06:17:27.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.065+0000 7f9adffff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0150b60 con 0x7f9ae006d840 2023-11-24T06:17:27.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.065+0000 7f9adffff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae013ace0 con 0x7f9ae0152bf0 2023-11-24T06:17:27.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.065+0000 7f9adeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9acc003a10 0x7f9ae013a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:27.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.066+0000 7f9adeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9acc003a10 0x7f9ae013a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37528/0 (socket says 172.21.15.5:37528) 2023-11-24T06:17:27.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.066+0000 7f9adeffd640 1 -- 172.21.15.5:0/2255585786 learned_addr learned my addr 172.21.15.5:0/2255585786 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:27.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.066+0000 7f9adeffd640 1 -- 172.21.15.5:0/2255585786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ae0152bf0 msgr2=0x7f9ae006d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:27.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.066+0000 7f9adeffd640 1 --2- 172.21.15.5:0/2255585786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ae0152bf0 0x7f9ae006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:27.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.066+0000 7f9adeffd640 1 -- 172.21.15.5:0/2255585786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ae006d840 msgr2=0x7f9ae0150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:27.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.066+0000 7f9adeffd640 1 --2- 172.21.15.5:0/2255585786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ae006d840 0x7f9ae0150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:27.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.066+0000 7f9adeffd640 1 -- 172.21.15.5:0/2255585786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae0150230 con 0x7f9acc003a10 2023-11-24T06:17:27.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.066+0000 7f9adeffd640 1 --2- 172.21.15.5:0/2255585786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9acc003a10 0x7f9ae013a7a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0008fd0 tx=0x7f9ad0001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:27.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.066+0000 7f9adcff9640 1 -- 172.21.15.5:0/2255585786 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0007050 con 0x7f9acc003a10 2023-11-24T06:17:27.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.067+0000 7f9adcff9640 1 -- 172.21.15.5:0/2255585786 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad00071b0 con 0x7f9acc003a10 2023-11-24T06:17:27.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.067+0000 7f9adcff9640 1 -- 172.21.15.5:0/2255585786 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0007520 con 0x7f9acc003a10 2023-11-24T06:17:27.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.067+0000 7f9adffff640 1 -- 172.21.15.5:0/2255585786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae0150510 con 0x7f9acc003a10 2023-11-24T06:17:27.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.067+0000 7f9adffff640 1 -- 172.21.15.5:0/2255585786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae0159d80 con 0x7f9acc003a10 2023-11-24T06:17:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.068+0000 7f9adffff640 1 -- 172.21.15.5:0/2255585786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae004e840 con 0x7f9acc003a10 2023-11-24T06:17:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.069+0000 7f9adcff9640 1 -- 172.21.15.5:0/2255585786 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ad00067d0 con 0x7f9acc003a10 2023-11-24T06:17:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.070+0000 7f9adcff9640 1 --2- 172.21.15.5:0/2255585786 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f9ac006add0 0x7f9ac006d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.070+0000 7f9adcff9640 1 -- 172.21.15.5:0/2255585786 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ad00ab740 con 0x7f9acc003a10 2023-11-24T06:17:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.070+0000 7f9ad7fff640 1 --2- 172.21.15.5:0/2255585786 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f9ac006add0 0x7f9ac006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:27.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.071+0000 7f9ad7fff640 1 --2- 172.21.15.5:0/2255585786 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f9ac006add0 0x7f9ac006d290 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8006e80 tx=0x7f9ac8006dd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:27.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.075+0000 7f9adcff9640 1 -- 172.21.15.5:0/2255585786 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ad0076760 con 0x7f9acc003a10 2023-11-24T06:17:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.306+0000 7f9adffff640 1 -- 172.21.15.5:0/2255585786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9ae006dc80 con 0x7f9acc003a10 2023-11-24T06:17:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.307+0000 7f9adcff9640 1 -- 172.21.15.5:0/2255585786 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9ad004b060 con 0x7f9acc003a10 2023-11-24T06:17:27.310 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T06:17:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.310+0000 7f9adffff640 1 -- 172.21.15.5:0/2255585786 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f9ac006add0 msgr2=0x7f9ac006d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.310+0000 7f9adffff640 1 --2- 172.21.15.5:0/2255585786 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f9ac006add0 0x7f9ac006d290 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8006e80 tx=0x7f9ac8006dd0 comp rx=0 tx=0).stop 2023-11-24T06:17:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.310+0000 7f9adffff640 1 -- 172.21.15.5:0/2255585786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9acc003a10 msgr2=0x7f9ae013a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.310+0000 7f9adffff640 1 --2- 172.21.15.5:0/2255585786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9acc003a10 0x7f9ae013a7a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0008fd0 tx=0x7f9ad0001230 comp rx=0 tx=0).stop 2023-11-24T06:17:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.310+0000 7f9adffff640 1 -- 172.21.15.5:0/2255585786 shutdown_connections 2023-11-24T06:17:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.310+0000 7f9adffff640 1 --2- 172.21.15.5:0/2255585786 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f9ac006add0 0x7f9ac006d290 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.310+0000 7f9adffff640 1 --2- 172.21.15.5:0/2255585786 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f9ae0152bf0 0x7f9ae006d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.310+0000 7f9adffff640 1 --2- 172.21.15.5:0/2255585786 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f9ae006d840 0x7f9ae0150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.310+0000 7f9adffff640 1 --2- 172.21.15.5:0/2255585786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9acc003a10 0x7f9ae013a7a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.310+0000 7f9adffff640 1 -- 172.21.15.5:0/2255585786 >> 172.21.15.5:0/2255585786 conn(0x7f9ae006a860 msgr2=0x7f9ae00b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.310+0000 7f9adffff640 1 -- 172.21.15.5:0/2255585786 shutdown_connections 2023-11-24T06:17:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:27.310+0000 7f9adffff640 1 -- 172.21.15.5:0/2255585786 wait complete. 2023-11-24T06:17:27.331 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T06:17:27.331 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T06:17:27.331 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi006.front.sepia.ceph.com 2023-11-24T06:17:27.331 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T06:17:27.331 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T06:17:27.331 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T06:17:27.331 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T06:17:27.331 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T06:17:27.331 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:17:27.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:27.332 DEBUG:teuthology.orchestra.run.smithi006:> ip addr 2023-11-24T06:17:27.394 INFO:teuthology.orchestra.run.smithi006.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T06:17:27.394 INFO:teuthology.orchestra.run.smithi006.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T06:17:27.394 INFO:teuthology.orchestra.run.smithi006.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T06:17:27.395 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:17:27.395 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 ::1/128 scope host 2023-11-24T06:17:27.395 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:17:27.395 INFO:teuthology.orchestra.run.smithi006.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:17:27.395 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e6 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:17:27.395 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f0 2023-11-24T06:17:27.395 INFO:teuthology.orchestra.run.smithi006.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:17:27.395 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:6b:fe:e7 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:17:27.395 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp7s0f1 2023-11-24T06:17:27.395 INFO:teuthology.orchestra.run.smithi006.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T06:17:27.395 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e4 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:17:27.395 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f0 2023-11-24T06:17:27.396 INFO:teuthology.orchestra.run.smithi006.stdout: inet 172.21.15.6/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T06:17:27.396 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft 42252sec preferred_lft 42252sec 2023-11-24T06:17:27.396 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::ec4:7aff:febd:15e4/64 scope link noprefixroute 2023-11-24T06:17:27.396 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:17:27.396 INFO:teuthology.orchestra.run.smithi006.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T06:17:27.396 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 0c:c4:7a:bd:15:e5 brd ff:ff:ff:ff:ff:ff 2023-11-24T06:17:27.396 INFO:teuthology.orchestra.run.smithi006.stdout: altname enp1s0f1 2023-11-24T06:17:27.396 INFO:teuthology.orchestra.run.smithi006.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T06:17:27.396 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 66:33:f4:1b:9a:5c brd ff:ff:ff:ff:ff:ff 2023-11-24T06:17:27.396 INFO:teuthology.orchestra.run.smithi006.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T06:17:27.396 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:17:27.396 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::248e:b2ff:fe4b:5ed7/64 scope link 2023-11-24T06:17:27.397 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:17:27.397 INFO:teuthology.orchestra.run.smithi006.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:17:27.397 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 9a:18:c2:c1:49:59 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:17:27.397 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::9818:c2ff:fec1:4959/64 scope link 2023-11-24T06:17:27.397 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:17:27.397 INFO:teuthology.orchestra.run.smithi006.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T06:17:27.397 INFO:teuthology.orchestra.run.smithi006.stdout: link/ether 66:33:f4:1b:9a:5c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:17:27.397 INFO:teuthology.orchestra.run.smithi006.stdout: inet6 fe80::6433:f4ff:fe1b:9a5c/64 scope link 2023-11-24T06:17:27.397 INFO:teuthology.orchestra.run.smithi006.stdout: valid_lft forever preferred_lft forever 2023-11-24T06:17:27.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:27.398 DEBUG:teuthology.orchestra.run.smithi006:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T06:17:27.495 INFO:teuthology.orchestra.run.smithi006.stdout:1 2023-11-24T06:17:27.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:27.499 DEBUG:teuthology.orchestra.run.smithi006:> route 2023-11-24T06:17:27.559 INFO:teuthology.orchestra.run.smithi006.stdout:Kernel IP routing table 2023-11-24T06:17:27.561 INFO:teuthology.orchestra.run.smithi006.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T06:17:27.561 INFO:teuthology.orchestra.run.smithi006.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T06:17:27.561 INFO:teuthology.orchestra.run.smithi006.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T06:17:27.561 INFO:teuthology.orchestra.run.smithi006.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T06:17:27.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:27.562 DEBUG:teuthology.orchestra.run.smithi006:> (cd / && exec bash -c ' 2023-11-24T06:17:27.562 DEBUG:teuthology.orchestra.run.smithi006:> set -e 2023-11-24T06:17:27.563 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T06:17:27.563 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T06:17:27.563 DEBUG:teuthology.orchestra.run.smithi006:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T06:17:27.563 DEBUG:teuthology.orchestra.run.smithi006:> ') 2023-11-24T06:17:27.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:27.767 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-24T06:17:27.821 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:17:27.822 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:17:27.823 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:17:27.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:27.823 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link set brx.0 up 2023-11-24T06:17:27.923 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-24T06:17:27.924 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:17:27.924 DEBUG:teuthology.orchestra.run.smithi006:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T06:17:27.982 INFO:teuthology.orchestra.run.smithi006.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T06:17:27.982 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T06:17:27.982 DEBUG:teuthology.orchestra.run.smithi006:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:17:28.038 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-24T06:17:28.139 DEBUG:teuthology.orchestra.run.smithi006:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:17:28.199 INFO:teuthology.orchestra.run.smithi006.stdout:/proc 2023-11-24T06:17:28.200 INFO:teuthology.orchestra.run.smithi006.stdout:/sys 2023-11-24T06:17:28.200 INFO:teuthology.orchestra.run.smithi006.stdout:/dev 2023-11-24T06:17:28.200 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/security 2023-11-24T06:17:28.200 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/shm 2023-11-24T06:17:28.200 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/pts 2023-11-24T06:17:28.200 INFO:teuthology.orchestra.run.smithi006.stdout:/run 2023-11-24T06:17:28.200 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/cgroup 2023-11-24T06:17:28.200 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/pstore 2023-11-24T06:17:28.200 INFO:teuthology.orchestra.run.smithi006.stdout:/ 2023-11-24T06:17:28.200 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/selinux 2023-11-24T06:17:28.200 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:17:28.200 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/hugepages 2023-11-24T06:17:28.201 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/mqueue 2023-11-24T06:17:28.201 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/debug 2023-11-24T06:17:28.201 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/tracing 2023-11-24T06:17:28.201 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/fuse/connections 2023-11-24T06:17:28.201 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/config 2023-11-24T06:17:28.201 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:17:28.201 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:17:28.201 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:17:28.201 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:17:28.201 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/1000 2023-11-24T06:17:28.201 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph 2023-11-24T06:17:28.201 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:17:28.201 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:17:28.202 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:17:28.202 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:17:28.202 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns 2023-11-24T06:17:28.202 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:17:28.202 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:17:28.202 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:17:28.202 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:17:28.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:28.204 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-24T06:17:28.264 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-24T06:17:28.265 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-24T06:17:28.270 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-24T06:17:28.355 DEBUG:teuthology.orchestra.run.smithi006:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:17:28.404 INFO:teuthology.orchestra.run.smithi006.stdout:/proc 2023-11-24T06:17:28.404 INFO:teuthology.orchestra.run.smithi006.stdout:/sys 2023-11-24T06:17:28.404 INFO:teuthology.orchestra.run.smithi006.stdout:/dev 2023-11-24T06:17:28.404 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/security 2023-11-24T06:17:28.404 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/shm 2023-11-24T06:17:28.405 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/pts 2023-11-24T06:17:28.405 INFO:teuthology.orchestra.run.smithi006.stdout:/run 2023-11-24T06:17:28.405 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/cgroup 2023-11-24T06:17:28.405 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/pstore 2023-11-24T06:17:28.405 INFO:teuthology.orchestra.run.smithi006.stdout:/ 2023-11-24T06:17:28.405 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/selinux 2023-11-24T06:17:28.405 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:17:28.405 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/hugepages 2023-11-24T06:17:28.405 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/mqueue 2023-11-24T06:17:28.405 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/debug 2023-11-24T06:17:28.406 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/tracing 2023-11-24T06:17:28.406 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/fuse/connections 2023-11-24T06:17:28.406 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/config 2023-11-24T06:17:28.406 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:17:28.406 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:17:28.406 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:17:28.406 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:17:28.406 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/1000 2023-11-24T06:17:28.406 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph 2023-11-24T06:17:28.406 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:17:28.406 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:17:28.406 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:17:28.407 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:17:28.407 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns 2023-11-24T06:17:28.407 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:17:28.407 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:17:28.407 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:17:28.407 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:17:28.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:28.408 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-24T06:17:28.476 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi006.stderr:2023-11-24T06:17:28.473+0000 7fe03d977480 -1 init, newargv = 0x55ae65891c90 newargc=15 2023-11-24T06:17:28.476 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi006.stderr:ceph-fuse[94670]: starting ceph client 2023-11-24T06:17:28.488 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi006.stderr:ceph-fuse[94670]: starting fuse 2023-11-24T06:17:29.553 DEBUG:teuthology.orchestra.run.smithi006:> sudo modprobe fuse 2023-11-24T06:17:29.663 DEBUG:teuthology.orchestra.run.smithi006:> cat /proc/self/mounts | awk '{print $2}' 2023-11-24T06:17:29.725 INFO:teuthology.orchestra.run.smithi006.stdout:/proc 2023-11-24T06:17:29.725 INFO:teuthology.orchestra.run.smithi006.stdout:/sys 2023-11-24T06:17:29.725 INFO:teuthology.orchestra.run.smithi006.stdout:/dev 2023-11-24T06:17:29.725 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/security 2023-11-24T06:17:29.725 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/shm 2023-11-24T06:17:29.725 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/pts 2023-11-24T06:17:29.725 INFO:teuthology.orchestra.run.smithi006.stdout:/run 2023-11-24T06:17:29.726 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/cgroup 2023-11-24T06:17:29.726 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/pstore 2023-11-24T06:17:29.726 INFO:teuthology.orchestra.run.smithi006.stdout:/ 2023-11-24T06:17:29.726 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/selinux 2023-11-24T06:17:29.726 INFO:teuthology.orchestra.run.smithi006.stdout:/proc/sys/fs/binfmt_misc 2023-11-24T06:17:29.726 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/hugepages 2023-11-24T06:17:29.726 INFO:teuthology.orchestra.run.smithi006.stdout:/dev/mqueue 2023-11-24T06:17:29.726 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/debug 2023-11-24T06:17:29.726 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/tracing 2023-11-24T06:17:29.726 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/fs/fuse/connections 2023-11-24T06:17:29.726 INFO:teuthology.orchestra.run.smithi006.stdout:/sys/kernel/config 2023-11-24T06:17:29.727 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-sysctl.service 2023-11-24T06:17:29.727 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-24T06:17:29.727 INFO:teuthology.orchestra.run.smithi006.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-24T06:17:29.727 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/nfs/rpc_pipefs 2023-11-24T06:17:29.727 INFO:teuthology.orchestra.run.smithi006.stdout:/run/user/1000 2023-11-24T06:17:29.727 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph 2023-11-24T06:17:29.727 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-4 2023-11-24T06:17:29.727 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-5 2023-11-24T06:17:29.727 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-6 2023-11-24T06:17:29.728 INFO:teuthology.orchestra.run.smithi006.stdout:/var/lib/ceph/osd/ceph-7 2023-11-24T06:17:29.728 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns 2023-11-24T06:17:29.728 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:17:29.728 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:17:29.728 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:17:29.728 INFO:teuthology.orchestra.run.smithi006.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:17:29.729 INFO:teuthology.orchestra.run.smithi006.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-24T06:17:29.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:29.730 DEBUG:teuthology.orchestra.run.smithi006:> ls /sys/fs/fuse/connections 2023-11-24T06:17:29.784 INFO:teuthology.orchestra.run.smithi006.stdout:41 2023-11-24T06:17:29.785 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-11-24T06:17:29.785 DEBUG:teuthology.orchestra.run.smithi006:> sudo stdin-killer -- python3 -c ' 2023-11-24T06:17:29.785 DEBUG:teuthology.orchestra.run.smithi006:> import glob 2023-11-24T06:17:29.785 DEBUG:teuthology.orchestra.run.smithi006:> import re 2023-11-24T06:17:29.785 DEBUG:teuthology.orchestra.run.smithi006:> import os 2023-11-24T06:17:29.785 DEBUG:teuthology.orchestra.run.smithi006:> import subprocess 2023-11-24T06:17:29.785 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-24T06:17:29.785 DEBUG:teuthology.orchestra.run.smithi006:> def _find_admin_socket(client_name): 2023-11-24T06:17:29.785 DEBUG:teuthology.orchestra.run.smithi006:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-24T06:17:29.786 DEBUG:teuthology.orchestra.run.smithi006:> files = glob.glob(asok_path) 2023-11-24T06:17:29.786 DEBUG:teuthology.orchestra.run.smithi006:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-24T06:17:29.786 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-24T06:17:29.786 DEBUG:teuthology.orchestra.run.smithi006:> # Given a non-glob path, it better be there 2023-11-24T06:17:29.786 DEBUG:teuthology.orchestra.run.smithi006:> if "*" not in asok_path: 2023-11-24T06:17:29.786 DEBUG:teuthology.orchestra.run.smithi006:> assert(len(files) == 1) 2023-11-24T06:17:29.786 DEBUG:teuthology.orchestra.run.smithi006:> return files[0] 2023-11-24T06:17:29.786 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-24T06:17:29.787 DEBUG:teuthology.orchestra.run.smithi006:> for f in files: 2023-11-24T06:17:29.787 DEBUG:teuthology.orchestra.run.smithi006:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-24T06:17:29.787 DEBUG:teuthology.orchestra.run.smithi006:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-24T06:17:29.787 DEBUG:teuthology.orchestra.run.smithi006:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-24T06:17:29.787 DEBUG:teuthology.orchestra.run.smithi006:> contents = proc_f.read() 2023-11-24T06:17:29.787 DEBUG:teuthology.orchestra.run.smithi006:> if mountpoint in contents: 2023-11-24T06:17:29.787 DEBUG:teuthology.orchestra.run.smithi006:> return f 2023-11-24T06:17:29.787 DEBUG:teuthology.orchestra.run.smithi006:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-24T06:17:29.787 DEBUG:teuthology.orchestra.run.smithi006:> 2023-11-24T06:17:29.787 DEBUG:teuthology.orchestra.run.smithi006:> print(_find_admin_socket("client.1")) 2023-11-24T06:17:29.787 DEBUG:teuthology.orchestra.run.smithi006:> ' 2023-11-24T06:17:29.970 INFO:teuthology.orchestra.run.smithi006.stdout:/var/run/ceph/ceph-client.1.94670.asok 2023-11-24T06:17:29.975 INFO:teuthology.orchestra.run.smithi006.stderr:2023-11-24T06:17:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-24T06:17:29.989 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.94670.asok 2023-11-24T06:17:29.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:29.989 DEBUG:teuthology.orchestra.run.smithi006:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.94670.asok status 2023-11-24T06:17:30.236 INFO:teuthology.orchestra.run.smithi006.stdout:{ 2023-11-24T06:17:30.237 INFO:teuthology.orchestra.run.smithi006.stdout: "metadata": { 2023-11-24T06:17:30.237 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_sha1": "a13ea5ff00cb4df7a9cc941166ddeb358881144c", 2023-11-24T06:17:30.237 INFO:teuthology.orchestra.run.smithi006.stdout: "ceph_version": "ceph version 18.0.0-7484-ga13ea5ff (a13ea5ff00cb4df7a9cc941166ddeb358881144c) reef (dev)", 2023-11-24T06:17:30.237 INFO:teuthology.orchestra.run.smithi006.stdout: "entity_id": "1", 2023-11-24T06:17:30.237 INFO:teuthology.orchestra.run.smithi006.stdout: "hostname": "smithi006", 2023-11-24T06:17:30.237 INFO:teuthology.orchestra.run.smithi006.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-24T06:17:30.237 INFO:teuthology.orchestra.run.smithi006.stdout: "pid": "94670", 2023-11-24T06:17:30.237 INFO:teuthology.orchestra.run.smithi006.stdout: "root": "/" 2023-11-24T06:17:30.237 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-24T06:17:30.237 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_count": 0, 2023-11-24T06:17:30.237 INFO:teuthology.orchestra.run.smithi006.stdout: "dentry_pinned_count": 0, 2023-11-24T06:17:30.237 INFO:teuthology.orchestra.run.smithi006.stdout: "id": 5008, 2023-11-24T06:17:30.238 INFO:teuthology.orchestra.run.smithi006.stdout: "inst": { 2023-11-24T06:17:30.238 INFO:teuthology.orchestra.run.smithi006.stdout: "name": { 2023-11-24T06:17:30.238 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "client", 2023-11-24T06:17:30.238 INFO:teuthology.orchestra.run.smithi006.stdout: "num": 5008 2023-11-24T06:17:30.238 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-24T06:17:30.238 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-24T06:17:30.238 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-24T06:17:30.238 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:17:30.238 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1775659461 2023-11-24T06:17:30.238 INFO:teuthology.orchestra.run.smithi006.stdout: } 2023-11-24T06:17:30.238 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-24T06:17:30.238 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": { 2023-11-24T06:17:30.239 INFO:teuthology.orchestra.run.smithi006.stdout: "type": "v1", 2023-11-24T06:17:30.239 INFO:teuthology.orchestra.run.smithi006.stdout: "addr": "192.168.0.1:0", 2023-11-24T06:17:30.239 INFO:teuthology.orchestra.run.smithi006.stdout: "nonce": 1775659461 2023-11-24T06:17:30.239 INFO:teuthology.orchestra.run.smithi006.stdout: }, 2023-11-24T06:17:30.239 INFO:teuthology.orchestra.run.smithi006.stdout: "inst_str": "client.5008 192.168.0.1:0/1775659461", 2023-11-24T06:17:30.239 INFO:teuthology.orchestra.run.smithi006.stdout: "addr_str": "192.168.0.1:0/1775659461", 2023-11-24T06:17:30.239 INFO:teuthology.orchestra.run.smithi006.stdout: "inode_count": 1, 2023-11-24T06:17:30.239 INFO:teuthology.orchestra.run.smithi006.stdout: "mds_epoch": 17, 2023-11-24T06:17:30.239 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch": 35, 2023-11-24T06:17:30.239 INFO:teuthology.orchestra.run.smithi006.stdout: "osd_epoch_barrier": 0, 2023-11-24T06:17:30.239 INFO:teuthology.orchestra.run.smithi006.stdout: "blocklisted": false, 2023-11-24T06:17:30.239 INFO:teuthology.orchestra.run.smithi006.stdout: "fs_name": "cephfs" 2023-11-24T06:17:30.239 INFO:teuthology.orchestra.run.smithi006.stdout:} 2023-11-24T06:17:30.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:30.252 DEBUG:teuthology.orchestra.run.smithi006:> stat --file-system '--printf=%T 2023-11-24T06:17:30.252 DEBUG:teuthology.orchestra.run.smithi006:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-24T06:17:30.307 INFO:teuthology.orchestra.run.smithi006.stdout:fuseblk 2023-11-24T06:17:30.307 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-24T06:17:30.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:30.308 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T06:17:30.391 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T06:17:30.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.631+0000 7fb497173640 1 Processor -- start 2023-11-24T06:17:30.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.631+0000 7fb497173640 1 -- start start 2023-11-24T06:17:30.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.632+0000 7fb497173640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb4900d1d90 0x7fb4900ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:30.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.633+0000 7fb497173640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4900d0fa0 0x7fb4900b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:30.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.633+0000 7fb497173640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4900c6bf0 con 0x7fb49015d3b0 2023-11-24T06:17:30.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.633+0000 7fb497173640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4900d13f0 con 0x7fb4900d1d90 2023-11-24T06:17:30.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.633+0000 7fb497173640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900cedc0 con 0x7fb4900d0fa0 2023-11-24T06:17:30.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.633+0000 7fb495970640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb4900d1d90 0x7fb4900ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:30.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.633+0000 7fb495970640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb4900d1d90 0x7fb4900ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:42428/0 (socket says 172.21.15.5:42428) 2023-11-24T06:17:30.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.633+0000 7fb496972640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4900d0fa0 0x7fb4900b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:30.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.633+0000 7fb495970640 1 -- 172.21.15.5:0/3206307209 learned_addr learned my addr 172.21.15.5:0/3206307209 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:30.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.634+0000 7fb496972640 1 -- 172.21.15.5:0/3206307209 >> v2:172.21.15.6:3300/0 conn(0x7fb4900d1d90 msgr2=0x7fb4900ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:30.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.634+0000 7fb496972640 1 --2- 172.21.15.5:0/3206307209 >> v2:172.21.15.6:3300/0 conn(0x7fb4900d1d90 0x7fb4900ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:30.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.634+0000 7fb496972640 1 -- 172.21.15.5:0/3206307209 >> v1:172.21.15.6:6789/0 conn(0x7fb49015d3b0 legacy=0x7fb4900d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:30.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.634+0000 7fb496972640 1 -- 172.21.15.5:0/3206307209 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900c6e40 con 0x7fb4900d0fa0 2023-11-24T06:17:30.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.634+0000 7fb495970640 1 --2- 172.21.15.5:0/3206307209 >> v2:172.21.15.6:3300/0 conn(0x7fb4900d1d90 0x7fb4900ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:17:30.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.636+0000 7fb496972640 1 --2- 172.21.15.5:0/3206307209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4900d0fa0 0x7fb4900b3ca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb488009d90 tx=0x7fb48802be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5711f5e55c3341f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:30.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.637+0000 7fb49516f640 1 -- 172.21.15.5:0/3206307209 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48802cbd0 con 0x7fb4900d0fa0 2023-11-24T06:17:30.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.637+0000 7fb49516f640 1 -- 172.21.15.5:0/3206307209 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48802cd30 con 0x7fb4900d0fa0 2023-11-24T06:17:30.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.637+0000 7fb49516f640 1 -- 172.21.15.5:0/3206307209 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48802cea0 con 0x7fb4900d0fa0 2023-11-24T06:17:30.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.637+0000 7fb497173640 1 -- 172.21.15.5:0/3206307209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4900d0fa0 msgr2=0x7fb4900b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.637+0000 7fb497173640 1 --2- 172.21.15.5:0/3206307209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4900d0fa0 0x7fb4900b3ca0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb488009d90 tx=0x7fb48802be20 comp rx=0 tx=0).stop 2023-11-24T06:17:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.637+0000 7fb497173640 1 -- 172.21.15.5:0/3206307209 shutdown_connections 2023-11-24T06:17:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.637+0000 7fb497173640 1 --2- 172.21.15.5:0/3206307209 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4900d0fa0 0x7fb4900b3ca0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.637+0000 7fb497173640 1 --2- 172.21.15.5:0/3206307209 >> v2:172.21.15.6:3300/0 conn(0x7fb4900d1d90 0x7fb4900ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.637+0000 7fb497173640 1 -- 172.21.15.5:0/3206307209 >> 172.21.15.5:0/3206307209 conn(0x7fb4900d30a0 msgr2=0x7fb4900d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.638+0000 7fb497173640 1 -- 172.21.15.5:0/3206307209 shutdown_connections 2023-11-24T06:17:30.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.638+0000 7fb497173640 1 -- 172.21.15.5:0/3206307209 wait complete. 2023-11-24T06:17:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.640+0000 7fb497173640 1 Processor -- start 2023-11-24T06:17:30.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.640+0000 7fb497173640 1 -- start start 2023-11-24T06:17:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.641+0000 7fb497173640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4900d0fa0 0x7fb4900cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.641+0000 7fb497173640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4900d1d90 0x7fb4900cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.641+0000 7fb497173640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015d3b0 0x7fb49014c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.641+0000 7fb497173640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49014bfa0 con 0x7fb49015d3b0 2023-11-24T06:17:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.641+0000 7fb497173640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900cbc70 con 0x7fb4900d0fa0 2023-11-24T06:17:30.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.641+0000 7fb497173640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900cbee0 con 0x7fb4900d1d90 2023-11-24T06:17:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.641+0000 7fb496972640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015d3b0 0x7fb49014c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.641+0000 7fb496972640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015d3b0 0x7fb49014c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37536/0 (socket says 172.21.15.5:37536) 2023-11-24T06:17:30.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.641+0000 7fb496972640 1 -- 172.21.15.5:0/3105018214 learned_addr learned my addr 172.21.15.5:0/3105018214 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.642+0000 7fb495970640 1 --2- 172.21.15.5:0/3105018214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4900d1d90 0x7fb4900cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.642+0000 7fb496171640 1 --2- 172.21.15.5:0/3105018214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4900d0fa0 0x7fb4900cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.642+0000 7fb496972640 1 -- 172.21.15.5:0/3105018214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4900d1d90 msgr2=0x7fb4900cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.642+0000 7fb496972640 1 --2- 172.21.15.5:0/3105018214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4900d1d90 0x7fb4900cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.642+0000 7fb496972640 1 -- 172.21.15.5:0/3105018214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4900d0fa0 msgr2=0x7fb4900cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:30.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.642+0000 7fb496972640 1 --2- 172.21.15.5:0/3105018214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4900d0fa0 0x7fb4900cc9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.642+0000 7fb496972640 1 -- 172.21.15.5:0/3105018214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4800094d0 con 0x7fb49015d3b0 2023-11-24T06:17:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.642+0000 7fb496972640 1 --2- 172.21.15.5:0/3105018214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015d3b0 0x7fb49014c1e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb48802c430 tx=0x7fb488004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.642+0000 7fb486ffd640 1 -- 172.21.15.5:0/3105018214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48802cea0 con 0x7fb49015d3b0 2023-11-24T06:17:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.642+0000 7fb486ffd640 1 -- 172.21.15.5:0/3105018214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48803e610 con 0x7fb49015d3b0 2023-11-24T06:17:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.643+0000 7fb486ffd640 1 -- 172.21.15.5:0/3105018214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48803ea00 con 0x7fb49015d3b0 2023-11-24T06:17:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.643+0000 7fb497173640 1 -- 172.21.15.5:0/3105018214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb488009a70 con 0x7fb49015d3b0 2023-11-24T06:17:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.643+0000 7fb497173640 1 -- 172.21.15.5:0/3105018214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4900cc4a0 con 0x7fb49015d3b0 2023-11-24T06:17:30.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.644+0000 7fb497173640 1 -- 172.21.15.5:0/3105018214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb49004e840 con 0x7fb49015d3b0 2023-11-24T06:17:30.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.645+0000 7fb486ffd640 1 -- 172.21.15.5:0/3105018214 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb48803b050 con 0x7fb49015d3b0 2023-11-24T06:17:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.646+0000 7fb486ffd640 1 --2- 172.21.15.5:0/3105018214 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb47406ae40 0x7fb47406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.646+0000 7fb486ffd640 1 -- 172.21.15.5:0/3105018214 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4880ab7e0 con 0x7fb49015d3b0 2023-11-24T06:17:30.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.647+0000 7fb496171640 1 --2- 172.21.15.5:0/3105018214 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb47406ae40 0x7fb47406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.647+0000 7fb496171640 1 --2- 172.21.15.5:0/3105018214 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb47406ae40 0x7fb47406d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb4900ce630 tx=0x7fb48c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:30.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.650+0000 7fb486ffd640 1 -- 172.21.15.5:0/3105018214 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb488076770 con 0x7fb49015d3b0 2023-11-24T06:17:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:30.889+0000 7fb497173640 1 -- 172.21.15.5:0/3105018214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb49006c780 con 0x7fb49015d3b0 2023-11-24T06:17:31.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:31.875+0000 7fb486ffd640 1 -- 172.21.15.5:0/3105018214 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 136+0+0 (secure 0 0 0) 0x7fb488009450 con 0x7fb49015d3b0 2023-11-24T06:17:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:31.879+0000 7fb497173640 1 -- 172.21.15.5:0/3105018214 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb47406ae40 msgr2=0x7fb47406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:31.879+0000 7fb497173640 1 --2- 172.21.15.5:0/3105018214 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb47406ae40 0x7fb47406d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb4900ce630 tx=0x7fb48c009040 comp rx=0 tx=0).stop 2023-11-24T06:17:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:31.879+0000 7fb497173640 1 -- 172.21.15.5:0/3105018214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015d3b0 msgr2=0x7fb49014c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:31.879+0000 7fb497173640 1 --2- 172.21.15.5:0/3105018214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015d3b0 0x7fb49014c1e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb48802c430 tx=0x7fb488004a40 comp rx=0 tx=0).stop 2023-11-24T06:17:31.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:31.879+0000 7fb497173640 1 -- 172.21.15.5:0/3105018214 shutdown_connections 2023-11-24T06:17:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:31.879+0000 7fb497173640 1 --2- 172.21.15.5:0/3105018214 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb47406ae40 0x7fb47406d300 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:31.879+0000 7fb497173640 1 --2- 172.21.15.5:0/3105018214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb4900d1d90 0x7fb4900cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:31.879+0000 7fb497173640 1 --2- 172.21.15.5:0/3105018214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb49015d3b0 0x7fb49014c1e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:31.879+0000 7fb497173640 1 --2- 172.21.15.5:0/3105018214 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb4900d0fa0 0x7fb4900cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:31.879+0000 7fb497173640 1 -- 172.21.15.5:0/3105018214 >> 172.21.15.5:0/3105018214 conn(0x7fb4900d30a0 msgr2=0x7fb4900d1680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:31.879+0000 7fb497173640 1 -- 172.21.15.5:0/3105018214 shutdown_connections 2023-11-24T06:17:31.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:31.879+0000 7fb497173640 1 -- 172.21.15.5:0/3105018214 wait complete. 2023-11-24T06:17:31.904 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T06:17:32.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.123+0000 7f76d249b640 1 Processor -- start 2023-11-24T06:17:32.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.124+0000 7f76d249b640 1 -- start start 2023-11-24T06:17:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.125+0000 7f76d249b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc15a090 0x7f76cc072150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.125+0000 7f76d249b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f76cc0abff0 0x7f76cc0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.125+0000 7f76d249b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76cc0ac760 con 0x7f76cc0b37d0 2023-11-24T06:17:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.125+0000 7f76d249b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76cc06e530 con 0x7f76cc0abff0 2023-11-24T06:17:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.125+0000 7f76d249b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc0b3c40 con 0x7f76cc15a090 2023-11-24T06:17:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.126+0000 7f76d0a38640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f76cc0b37d0 0x7f76cc0704b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35890/0 (socket says 172.21.15.5:35890) 2023-11-24T06:17:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.126+0000 7f76d0a38640 1 -- 172.21.15.5:0/480278611 learned_addr learned my addr 172.21.15.5:0/480278611 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.126+0000 7f76cbfff640 1 --2- 172.21.15.5:0/480278611 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc15a090 0x7f76cc072150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.126+0000 7f76c3fff640 1 --2- 172.21.15.5:0/480278611 >> v2:172.21.15.6:3300/0 conn(0x7f76cc0abff0 0x7f76cc0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.126+0000 7f76cb7fe640 1 -- 172.21.15.5:0/480278611 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855836260 0 0) 0x7f76cc0ac760 con 0x7f76cc0b37d0 2023-11-24T06:17:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.126+0000 7f76cb7fe640 1 -- 172.21.15.5:0/480278611 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76b8003520 con 0x7f76cc0b37d0 2023-11-24T06:17:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.127+0000 7f76cbfff640 1 -- 172.21.15.5:0/480278611 >> v2:172.21.15.6:3300/0 conn(0x7f76cc0abff0 msgr2=0x7f76cc0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.127+0000 7f76cbfff640 1 --2- 172.21.15.5:0/480278611 >> v2:172.21.15.6:3300/0 conn(0x7f76cc0abff0 0x7f76cc0b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.127+0000 7f76cbfff640 1 -- 172.21.15.5:0/480278611 >> v1:172.21.15.5:6789/0 conn(0x7f76cc0b37d0 legacy=0x7f76cc0704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.127+0000 7f76cbfff640 1 -- 172.21.15.5:0/480278611 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc15a7f0 con 0x7f76cc15a090 2023-11-24T06:17:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.128+0000 7f76cbfff640 1 --2- 172.21.15.5:0/480278611 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc15a090 0x7f76cc072150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0097a0 tx=0x7f76bc02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32b7ed16c77e04f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.129+0000 7f76cb7fe640 1 -- 172.21.15.5:0/480278611 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc02d040 con 0x7f76cc15a090 2023-11-24T06:17:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.129+0000 7f76cb7fe640 1 -- 172.21.15.5:0/480278611 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76bc031810 con 0x7f76cc15a090 2023-11-24T06:17:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.129+0000 7f76cb7fe640 1 -- 172.21.15.5:0/480278611 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc031b80 con 0x7f76cc15a090 2023-11-24T06:17:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.129+0000 7f76d249b640 1 -- 172.21.15.5:0/480278611 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc15a090 msgr2=0x7f76cc072150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.129+0000 7f76d249b640 1 --2- 172.21.15.5:0/480278611 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc15a090 0x7f76cc072150 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0097a0 tx=0x7f76bc02b8b0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.130+0000 7f76d249b640 1 -- 172.21.15.5:0/480278611 shutdown_connections 2023-11-24T06:17:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.130+0000 7f76d249b640 1 --2- 172.21.15.5:0/480278611 >> v2:172.21.15.6:3300/0 conn(0x7f76cc0abff0 0x7f76cc0b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.130+0000 7f76d249b640 1 --2- 172.21.15.5:0/480278611 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc15a090 0x7f76cc072150 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.130+0000 7f76d249b640 1 -- 172.21.15.5:0/480278611 >> 172.21.15.5:0/480278611 conn(0x7f76cc0b0d80 msgr2=0x7f76cc0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.130+0000 7f76d249b640 1 -- 172.21.15.5:0/480278611 shutdown_connections 2023-11-24T06:17:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.130+0000 7f76d249b640 1 -- 172.21.15.5:0/480278611 wait complete. 2023-11-24T06:17:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.132+0000 7f76d249b640 1 Processor -- start 2023-11-24T06:17:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.133+0000 7f76d249b640 1 -- start start 2023-11-24T06:17:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.134+0000 7f76d249b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76cc0abff0 0x7f76cc0b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.134+0000 7f76d249b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc0b37d0 0x7f76cc147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.134+0000 7f76d249b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc15a090 0x7f76cc147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.134+0000 7f76d249b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc157010 con 0x7f76cc0abff0 2023-11-24T06:17:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.134+0000 7f76d249b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc14c320 con 0x7f76cc15a090 2023-11-24T06:17:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.134+0000 7f76d249b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc148140 con 0x7f76cc0b37d0 2023-11-24T06:17:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.134+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76cc0abff0 0x7f76cc0b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.134+0000 7f76cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76cc0abff0 0x7f76cc0b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37548/0 (socket says 172.21.15.5:37548) 2023-11-24T06:17:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.134+0000 7f76cbfff640 1 -- 172.21.15.5:0/636470103 learned_addr learned my addr 172.21.15.5:0/636470103 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.135+0000 7f76d0a38640 1 --2- 172.21.15.5:0/636470103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc15a090 0x7f76cc147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.135+0000 7f76c3fff640 1 --2- 172.21.15.5:0/636470103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc0b37d0 0x7f76cc147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.135+0000 7f76cbfff640 1 -- 172.21.15.5:0/636470103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc0b37d0 msgr2=0x7f76cc147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.135+0000 7f76cbfff640 1 --2- 172.21.15.5:0/636470103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc0b37d0 0x7f76cc147040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.135+0000 7f76cbfff640 1 -- 172.21.15.5:0/636470103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc15a090 msgr2=0x7f76cc147580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.135+0000 7f76cbfff640 1 --2- 172.21.15.5:0/636470103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc15a090 0x7f76cc147580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.135+0000 7f76cbfff640 1 -- 172.21.15.5:0/636470103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc009400 con 0x7f76cc0abff0 2023-11-24T06:17:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.135+0000 7f76cbfff640 1 --2- 172.21.15.5:0/636470103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76cc0abff0 0x7f76cc0b00d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0098c0 tx=0x7f76bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.135+0000 7f76c97fa640 1 -- 172.21.15.5:0/636470103 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc006e10 con 0x7f76cc0abff0 2023-11-24T06:17:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.135+0000 7f76c97fa640 1 -- 172.21.15.5:0/636470103 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76bc031e20 con 0x7f76cc0abff0 2023-11-24T06:17:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.135+0000 7f76c97fa640 1 -- 172.21.15.5:0/636470103 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc03bd00 con 0x7f76cc0abff0 2023-11-24T06:17:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.135+0000 7f76d249b640 1 -- 172.21.15.5:0/636470103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76cc156950 con 0x7f76cc0abff0 2023-11-24T06:17:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.136+0000 7f76d249b640 1 -- 172.21.15.5:0/636470103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc148420 con 0x7f76cc0abff0 2023-11-24T06:17:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.137+0000 7f76d249b640 1 -- 172.21.15.5:0/636470103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76cc157280 con 0x7f76cc0abff0 2023-11-24T06:17:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.138+0000 7f76c97fa640 1 -- 172.21.15.5:0/636470103 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f76bc043070 con 0x7f76cc0abff0 2023-11-24T06:17:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.139+0000 7f76c97fa640 1 --2- 172.21.15.5:0/636470103 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f76ac06ae40 0x7f76ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.139+0000 7f76c97fa640 1 -- 172.21.15.5:0/636470103 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76bc0ab3c0 con 0x7f76cc0abff0 2023-11-24T06:17:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.139+0000 7f76c3fff640 1 --2- 172.21.15.5:0/636470103 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f76ac06ae40 0x7f76ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.140+0000 7f76c3fff640 1 --2- 172.21.15.5:0/636470103 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f76ac06ae40 0x7f76ac06d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0b3260 tx=0x7f76b4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:32.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.144+0000 7f76c97fa640 1 -- 172.21.15.5:0/636470103 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f76bc076590 con 0x7f76cc0abff0 2023-11-24T06:17:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.368+0000 7f76d249b640 1 -- 172.21.15.5:0/636470103 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f76cc0716f0 con 0x7f76ac06ae40 2023-11-24T06:17:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.371+0000 7f76c97fa640 1 -- 172.21.15.5:0/636470103 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f76cc0716f0 con 0x7f76ac06ae40 2023-11-24T06:17:32.374 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2023-11-24T06:17:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-11-24T06:17:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephfs" 2023-11-24T06:17:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-11-24T06:17:32.374 INFO:teuthology.orchestra.run.smithi005.stdout:] 2023-11-24T06:17:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.374+0000 7f76d249b640 1 -- 172.21.15.5:0/636470103 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f76ac06ae40 msgr2=0x7f76ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.374+0000 7f76d249b640 1 --2- 172.21.15.5:0/636470103 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f76ac06ae40 0x7f76ac06d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f76cc0b3260 tx=0x7f76b4008040 comp rx=0 tx=0).stop 2023-11-24T06:17:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.374+0000 7f76d249b640 1 -- 172.21.15.5:0/636470103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76cc0abff0 msgr2=0x7f76cc0b00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.374+0000 7f76d249b640 1 --2- 172.21.15.5:0/636470103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76cc0abff0 0x7f76cc0b00d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f76bc0098c0 tx=0x7f76bc0046c0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.374+0000 7f76d249b640 1 -- 172.21.15.5:0/636470103 shutdown_connections 2023-11-24T06:17:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.374+0000 7f76d249b640 1 --2- 172.21.15.5:0/636470103 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f76ac06ae40 0x7f76ac06d300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.374+0000 7f76d249b640 1 --2- 172.21.15.5:0/636470103 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f76cc0b37d0 0x7f76cc147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.374+0000 7f76d249b640 1 --2- 172.21.15.5:0/636470103 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f76cc15a090 0x7f76cc147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.374+0000 7f76d249b640 1 --2- 172.21.15.5:0/636470103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76cc0abff0 0x7f76cc0b00d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.374+0000 7f76d249b640 1 -- 172.21.15.5:0/636470103 >> 172.21.15.5:0/636470103 conn(0x7f76cc0b0d80 msgr2=0x7f76cc150130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.374+0000 7f76d249b640 1 -- 172.21.15.5:0/636470103 shutdown_connections 2023-11-24T06:17:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.374+0000 7f76d249b640 1 -- 172.21.15.5:0/636470103 wait complete. 2023-11-24T06:17:32.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T06:17:32.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.626+0000 7f8df1ee7640 1 Processor -- start 2023-11-24T06:17:32.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.626+0000 7f8df1ee7640 1 -- start start 2023-11-24T06:17:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.628+0000 7f8df1ee7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8dec15b350 0x7f8dec0abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.628+0000 7f8df1ee7640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dec15b730 con 0x7f8dec0adb60 2023-11-24T06:17:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.628+0000 7f8df1ee7640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dec0ac740 con 0x7f8dec09cb30 2023-11-24T06:17:32.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.628+0000 7f8df1ee7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dec0ac460 con 0x7f8dec15b350 2023-11-24T06:17:32.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.628+0000 7f8deb7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8dec15b350 0x7f8dec0abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:32.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.628+0000 7f8deb7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8dec15b350 0x7f8dec0abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37560/0 (socket says 172.21.15.5:37560) 2023-11-24T06:17:32.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.628+0000 7f8debfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8dec0adb60 0x7f8dec0adf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35896/0 (socket says 172.21.15.5:35896) 2023-11-24T06:17:32.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.628+0000 7f8deb7fe640 1 -- 172.21.15.5:0/2506340846 learned_addr learned my addr 172.21.15.5:0/2506340846 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:32.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.629+0000 7f8deb7fe640 1 -- 172.21.15.5:0/2506340846 >> v1:172.21.15.6:6789/0 conn(0x7f8dec09cb30 legacy=0x7f8dec0b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:32.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.629+0000 7f8deb7fe640 1 -- 172.21.15.5:0/2506340846 >> v1:172.21.15.5:6789/0 conn(0x7f8dec0adb60 legacy=0x7f8dec0adf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.629+0000 7f8deb7fe640 1 -- 172.21.15.5:0/2506340846 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dec0b2a50 con 0x7f8dec15b350 2023-11-24T06:17:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.629+0000 7f8dea7fc640 1 -- 172.21.15.5:0/2506340846 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317076160 0 0) 0x7f8de4001110 con 0x7f8dec0adb60 2023-11-24T06:17:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.630+0000 7f8deb7fe640 1 --2- 172.21.15.5:0/2506340846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dec15b350 0x7f8dec0abe80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc009870 tx=0x7f8ddc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef80fa9c3b8a0ce6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.630+0000 7f8dea7fc640 1 -- 172.21.15.5:0/2506340846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc030860 con 0x7f8dec15b350 2023-11-24T06:17:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.630+0000 7f8dea7fc640 1 -- 172.21.15.5:0/2506340846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ddc0309c0 con 0x7f8dec15b350 2023-11-24T06:17:32.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.631+0000 7f8dea7fc640 1 -- 172.21.15.5:0/2506340846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc030d50 con 0x7f8dec15b350 2023-11-24T06:17:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.631+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2506340846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dec15b350 msgr2=0x7f8dec0abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.631+0000 7f8df1ee7640 1 --2- 172.21.15.5:0/2506340846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dec15b350 0x7f8dec0abe80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc009870 tx=0x7f8ddc02b9a0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.631+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2506340846 shutdown_connections 2023-11-24T06:17:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.631+0000 7f8df1ee7640 1 --2- 172.21.15.5:0/2506340846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dec15b350 0x7f8dec0abe80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.631+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2506340846 >> 172.21.15.5:0/2506340846 conn(0x7f8dec0b0ba0 msgr2=0x7f8dec0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.631+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2506340846 shutdown_connections 2023-11-24T06:17:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.631+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2506340846 wait complete. 2023-11-24T06:17:32.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.633+0000 7f8df1ee7640 1 Processor -- start 2023-11-24T06:17:32.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.633+0000 7f8df1ee7640 1 -- start start 2023-11-24T06:17:32.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.634+0000 7f8df1ee7640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dec09cb30 0x7f8dec149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:32.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.634+0000 7f8df1ee7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dec0adb60 0x7f8dec14d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:32.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.634+0000 7f8df1ee7640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dec15b350 0x7f8dec13f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:32.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.634+0000 7f8df1ee7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec14a660 con 0x7f8dec0adb60 2023-11-24T06:17:32.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.634+0000 7f8df1ee7640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec14a380 con 0x7f8dec09cb30 2023-11-24T06:17:32.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.634+0000 7f8df1ee7640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec15cac0 con 0x7f8dec15b350 2023-11-24T06:17:32.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.634+0000 7f8deaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dec0adb60 0x7f8dec14d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:32.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.635+0000 7f8deaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dec0adb60 0x7f8dec14d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37576/0 (socket says 172.21.15.5:37576) 2023-11-24T06:17:32.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.635+0000 7f8deaffd640 1 -- 172.21.15.5:0/2588976638 learned_addr learned my addr 172.21.15.5:0/2588976638 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:32.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.635+0000 7f8deb7fe640 1 --2- 172.21.15.5:0/2588976638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dec09cb30 0x7f8dec149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:32.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.635+0000 7f8debfff640 1 --2- 172.21.15.5:0/2588976638 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dec15b350 0x7f8dec13f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:32.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.635+0000 7f8deaffd640 1 -- 172.21.15.5:0/2588976638 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dec15b350 msgr2=0x7f8dec13f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:32.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.635+0000 7f8deaffd640 1 --2- 172.21.15.5:0/2588976638 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dec15b350 0x7f8dec13f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.635+0000 7f8deaffd640 1 -- 172.21.15.5:0/2588976638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dec09cb30 msgr2=0x7f8dec149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:32.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.635+0000 7f8deaffd640 1 --2- 172.21.15.5:0/2588976638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dec09cb30 0x7f8dec149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.635+0000 7f8deaffd640 1 -- 172.21.15.5:0/2588976638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc0094d0 con 0x7f8dec0adb60 2023-11-24T06:17:32.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.635+0000 7f8deaffd640 1 --2- 172.21.15.5:0/2588976638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dec0adb60 0x7f8dec14d1b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8de000c7c0 tx=0x7f8de000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:32.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.636+0000 7f8dd3fff640 1 -- 172.21.15.5:0/2588976638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de000aa50 con 0x7f8dec0adb60 2023-11-24T06:17:32.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.636+0000 7f8dd3fff640 1 -- 172.21.15.5:0/2588976638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de000abb0 con 0x7f8dec0adb60 2023-11-24T06:17:32.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.636+0000 7f8dd3fff640 1 -- 172.21.15.5:0/2588976638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0012420 con 0x7f8dec0adb60 2023-11-24T06:17:32.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.636+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2588976638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dec15cda0 con 0x7f8dec0adb60 2023-11-24T06:17:32.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.636+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2588976638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dec149030 con 0x7f8dec0adb60 2023-11-24T06:17:32.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.637+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2588976638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dec15c920 con 0x7f8dec0adb60 2023-11-24T06:17:32.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.638+0000 7f8dd3fff640 1 -- 172.21.15.5:0/2588976638 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8de0012580 con 0x7f8dec0adb60 2023-11-24T06:17:32.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.640+0000 7f8dd3fff640 1 --2- 172.21.15.5:0/2588976638 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8dd806ae70 0x7f8dd806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:32.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.640+0000 7f8dd3fff640 1 -- 172.21.15.5:0/2588976638 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8de0089690 con 0x7f8dec0adb60 2023-11-24T06:17:32.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.640+0000 7f8deb7fe640 1 --2- 172.21.15.5:0/2588976638 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8dd806ae70 0x7f8dd806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:32.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.641+0000 7f8deb7fe640 1 --2- 172.21.15.5:0/2588976638 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8dd806ae70 0x7f8dd806d330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc036000 tx=0x7f8ddc03b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:32.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.645+0000 7f8dd3fff640 1 -- 172.21.15.5:0/2588976638 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8dec15c920 con 0x7f8dec0adb60 2023-11-24T06:17:32.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.839+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2588976638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8dec15c920 con 0x7f8dec0adb60 2023-11-24T06:17:32.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.840+0000 7f8dd3fff640 1 -- 172.21.15.5:0/2588976638 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8dec15c920 con 0x7f8dec0adb60 2023-11-24T06:17:32.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.843+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2588976638 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8dd806ae70 msgr2=0x7f8dd806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:32.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.843+0000 7f8df1ee7640 1 --2- 172.21.15.5:0/2588976638 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8dd806ae70 0x7f8dd806d330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc036000 tx=0x7f8ddc03b040 comp rx=0 tx=0).stop 2023-11-24T06:17:32.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.843+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2588976638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dec0adb60 msgr2=0x7f8dec14d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:32.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.843+0000 7f8df1ee7640 1 --2- 172.21.15.5:0/2588976638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dec0adb60 0x7f8dec14d1b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8de000c7c0 tx=0x7f8de000cc80 comp rx=0 tx=0).stop 2023-11-24T06:17:32.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.843+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2588976638 shutdown_connections 2023-11-24T06:17:32.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.843+0000 7f8df1ee7640 1 --2- 172.21.15.5:0/2588976638 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8dd806ae70 0x7f8dd806d330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.843+0000 7f8df1ee7640 1 --2- 172.21.15.5:0/2588976638 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8dec15b350 0x7f8dec13f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.843+0000 7f8df1ee7640 1 --2- 172.21.15.5:0/2588976638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8dec0adb60 0x7f8dec14d1b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.843+0000 7f8df1ee7640 1 --2- 172.21.15.5:0/2588976638 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8dec09cb30 0x7f8dec149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:32.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.843+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2588976638 >> 172.21.15.5:0/2588976638 conn(0x7f8dec0b0ba0 msgr2=0x7f8dec0b00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:32.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.843+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2588976638 shutdown_connections 2023-11-24T06:17:32.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:32.843+0000 7f8df1ee7640 1 -- 172.21.15.5:0/2588976638 wait complete. 2023-11-24T06:17:32.865 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol7562 --mode=777 2023-11-24T06:17:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.088+0000 7fe32b59e640 1 Processor -- start 2023-11-24T06:17:33.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.088+0000 7fe32b59e640 1 -- start start 2023-11-24T06:17:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.090+0000 7fe32b59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe32c0b2960 0x7fe32c06d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.090+0000 7fe32b59e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe32c0b2d40 0x7fe32c0b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.090+0000 7fe32b59e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c072610 con 0x7fe32c06c310 2023-11-24T06:17:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.090+0000 7fe32b59e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe32c1596b0 con 0x7fe32c0b2960 2023-11-24T06:17:33.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.090+0000 7fe32b59e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe32c06d3b0 con 0x7fe32c0b2d40 2023-11-24T06:17:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.090+0000 7fe32a59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe32c0b2960 0x7fe32c06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.091+0000 7fe32a59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe32c0b2960 0x7fe32c06d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37054/0 (socket says 172.21.15.5:37054) 2023-11-24T06:17:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.091+0000 7fe32a59c640 1 -- 172.21.15.5:0/1947474732 learned_addr learned my addr 172.21.15.5:0/1947474732 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.091+0000 7fe32ad9d640 1 --2- 172.21.15.5:0/1947474732 >> v2:172.21.15.6:3300/0 conn(0x7fe32c0b2d40 0x7fe32c0b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.091+0000 7fe32a59c640 1 -- 172.21.15.5:0/1947474732 >> v2:172.21.15.6:3300/0 conn(0x7fe32c0b2d40 msgr2=0x7fe32c0b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:33.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.091+0000 7fe32a59c640 1 --2- 172.21.15.5:0/1947474732 >> v2:172.21.15.6:3300/0 conn(0x7fe32c0b2d40 0x7fe32c0b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.091+0000 7fe32a59c640 1 -- 172.21.15.5:0/1947474732 >> v1:172.21.15.6:6789/0 conn(0x7fe32c06c310 legacy=0x7fe32c071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:33.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.091+0000 7fe32a59c640 1 -- 172.21.15.5:0/1947474732 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c06c7a0 con 0x7fe32c0b2960 2023-11-24T06:17:33.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.092+0000 7fe32a59c640 1 --2- 172.21.15.5:0/1947474732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe32c0b2960 0x7fe32c06d840 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009870 tx=0x7fe31c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983334c932d2c669 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:33.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.093+0000 7fe329d9b640 1 -- 172.21.15.5:0/1947474732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c031870 con 0x7fe32c0b2960 2023-11-24T06:17:33.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.093+0000 7fe329d9b640 1 -- 172.21.15.5:0/1947474732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31c0319d0 con 0x7fe32c0b2960 2023-11-24T06:17:33.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.093+0000 7fe329d9b640 1 -- 172.21.15.5:0/1947474732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c031d60 con 0x7fe32c0b2960 2023-11-24T06:17:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.093+0000 7fe32b59e640 1 -- 172.21.15.5:0/1947474732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe32c0b2960 msgr2=0x7fe32c06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.093+0000 7fe32b59e640 1 --2- 172.21.15.5:0/1947474732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe32c0b2960 0x7fe32c06d840 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009870 tx=0x7fe31c02b910 comp rx=0 tx=0).stop 2023-11-24T06:17:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.093+0000 7fe32b59e640 1 -- 172.21.15.5:0/1947474732 shutdown_connections 2023-11-24T06:17:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.093+0000 7fe32b59e640 1 --2- 172.21.15.5:0/1947474732 >> v2:172.21.15.6:3300/0 conn(0x7fe32c0b2d40 0x7fe32c0b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.093+0000 7fe32b59e640 1 --2- 172.21.15.5:0/1947474732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe32c0b2960 0x7fe32c06d840 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.093+0000 7fe32b59e640 1 -- 172.21.15.5:0/1947474732 >> 172.21.15.5:0/1947474732 conn(0x7fe32c0cec10 msgr2=0x7fe32c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:33.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.093+0000 7fe32b59e640 1 -- 172.21.15.5:0/1947474732 shutdown_connections 2023-11-24T06:17:33.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.093+0000 7fe32b59e640 1 -- 172.21.15.5:0/1947474732 wait complete. 2023-11-24T06:17:33.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.095+0000 7fe32b59e640 1 Processor -- start 2023-11-24T06:17:33.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.095+0000 7fe32b59e640 1 -- start start 2023-11-24T06:17:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.096+0000 7fe32b59e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe32c0b2960 0x7fe32c1589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.096+0000 7fe32b59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe32c0b2d40 0x7fe32c14ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.096+0000 7fe32b59e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe32c14d350 0x7fe32c153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.096+0000 7fe32b59e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c14e590 con 0x7fe32c0b2d40 2023-11-24T06:17:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.096+0000 7fe32b59e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c158ef0 con 0x7fe32c0b2960 2023-11-24T06:17:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.096+0000 7fe32b59e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c14d980 con 0x7fe32c14d350 2023-11-24T06:17:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.097+0000 7fe323fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe32c0b2d40 0x7fe32c14ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.097+0000 7fe323fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe32c0b2d40 0x7fe32c14ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37056/0 (socket says 172.21.15.5:37056) 2023-11-24T06:17:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.097+0000 7fe323fff640 1 -- 172.21.15.5:0/156186541 learned_addr learned my addr 172.21.15.5:0/156186541 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.097+0000 7fe32a59c640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe32c0b2960 0x7fe32c1589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.097+0000 7fe32ad9d640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe32c14d350 0x7fe32c153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.097+0000 7fe323fff640 1 -- 172.21.15.5:0/156186541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe32c14d350 msgr2=0x7fe32c153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.097+0000 7fe323fff640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe32c14d350 0x7fe32c153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.097+0000 7fe323fff640 1 -- 172.21.15.5:0/156186541 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe32c0b2960 msgr2=0x7fe32c1589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.097+0000 7fe323fff640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe32c0b2960 0x7fe32c1589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.097+0000 7fe323fff640 1 -- 172.21.15.5:0/156186541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c0094d0 con 0x7fe32c0b2d40 2023-11-24T06:17:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.097+0000 7fe32ad9d640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe32c14d350 0x7fe32c153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.098+0000 7fe323fff640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe32c0b2d40 0x7fe32c14ce10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe31400b420 tx=0x7fe31400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.098+0000 7fe322ffd640 1 -- 172.21.15.5:0/156186541 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314011830 con 0x7fe32c0b2d40 2023-11-24T06:17:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.098+0000 7fe322ffd640 1 -- 172.21.15.5:0/156186541 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe314011990 con 0x7fe32c0b2d40 2023-11-24T06:17:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.098+0000 7fe322ffd640 1 -- 172.21.15.5:0/156186541 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314011d40 con 0x7fe32c0b2d40 2023-11-24T06:17:33.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.098+0000 7fe32b59e640 1 -- 172.21.15.5:0/156186541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32c14dc60 con 0x7fe32c0b2d40 2023-11-24T06:17:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.098+0000 7fe32b59e640 1 -- 172.21.15.5:0/156186541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c14e220 con 0x7fe32c0b2d40 2023-11-24T06:17:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.099+0000 7fe32b59e640 1 -- 172.21.15.5:0/156186541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32c14e740 con 0x7fe32c0b2d40 2023-11-24T06:17:33.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.100+0000 7fe322ffd640 1 -- 172.21.15.5:0/156186541 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe31401b460 con 0x7fe32c0b2d40 2023-11-24T06:17:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.101+0000 7fe322ffd640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe30c06ae40 0x7fe30c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.101+0000 7fe322ffd640 1 -- 172.21.15.5:0/156186541 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe314089180 con 0x7fe32c0b2d40 2023-11-24T06:17:33.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.102+0000 7fe32a59c640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe30c06ae40 0x7fe30c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:33.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.102+0000 7fe32a59c640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe30c06ae40 0x7fe30c06d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe31c036000 tx=0x7fe31c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:33.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.106+0000 7fe322ffd640 1 -- 172.21.15.5:0/156186541 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe32c14e740 con 0x7fe32c0b2d40 2023-11-24T06:17:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.326+0000 7fe32b59e640 1 -- 172.21.15.5:0/156186541 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol7562", "mode": "777", "target": ["mon-mgr", ""]}) v1 -- 0x7fe32c06d040 con 0x7fe30c06ae40 2023-11-24T06:17:33.338 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:17:33.335+0000 7f399c0d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:17:33.338 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:17:33.335+0000 7f399c0d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:17:33.338 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:17:33.335+0000 7f399c0d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:17:33.339 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:17:33.335+0000 7f399c0d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:17:33.339 INFO:tasks.ceph.mgr.y.smithi005.stderr:2023-11-24T06:17:33.335+0000 7f399c0d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T06:17:33.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.384+0000 7fe322ffd640 1 -- 172.21.15.5:0/156186541 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe31c031bc0 con 0x7fe30c06ae40 2023-11-24T06:17:33.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.387+0000 7fe32b59e640 1 -- 172.21.15.5:0/156186541 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe30c06ae40 msgr2=0x7fe30c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:33.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.387+0000 7fe32b59e640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe30c06ae40 0x7fe30c06d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe31c036000 tx=0x7fe31c03c040 comp rx=0 tx=0).stop 2023-11-24T06:17:33.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.387+0000 7fe32b59e640 1 -- 172.21.15.5:0/156186541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe32c0b2d40 msgr2=0x7fe32c14ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:33.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.387+0000 7fe32b59e640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe32c0b2d40 0x7fe32c14ce10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe31400b420 tx=0x7fe31400b8e0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.387+0000 7fe32b59e640 1 -- 172.21.15.5:0/156186541 shutdown_connections 2023-11-24T06:17:33.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.387+0000 7fe32b59e640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe30c06ae40 0x7fe30c06d300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.387+0000 7fe32b59e640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe32c14d350 0x7fe32c153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.387+0000 7fe32b59e640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe32c0b2d40 0x7fe32c14ce10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.387+0000 7fe32b59e640 1 --2- 172.21.15.5:0/156186541 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe32c0b2960 0x7fe32c1589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.387+0000 7fe32b59e640 1 -- 172.21.15.5:0/156186541 >> 172.21.15.5:0/156186541 conn(0x7fe32c0cec10 msgr2=0x7fe32c1597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:33.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.387+0000 7fe32b59e640 1 -- 172.21.15.5:0/156186541 shutdown_connections 2023-11-24T06:17:33.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.387+0000 7fe32b59e640 1 -- 172.21.15.5:0/156186541 wait complete. 2023-11-24T06:17:33.410 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol7562 2023-11-24T06:17:33.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.607+0000 7fc46410f640 1 Processor -- start 2023-11-24T06:17:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.608+0000 7fc46410f640 1 -- start start 2023-11-24T06:17:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.609+0000 7fc46410f640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc45c0acd70 0x7fc45c06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.609+0000 7fc46410f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc45c070490 0x7fc45c070900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.609+0000 7fc46410f640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc45c06d7a0 con 0x7fc45c0ae490 2023-11-24T06:17:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.609+0000 7fc46410f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45c15a5b0 con 0x7fc45c070490 2023-11-24T06:17:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.609+0000 7fc46410f640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45c06ce70 con 0x7fc45c0acd70 2023-11-24T06:17:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.610+0000 7fc4626ac640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc45c070490 0x7fc45c070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.610+0000 7fc4626ac640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc45c070490 0x7fc45c070900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37074/0 (socket says 172.21.15.5:37074) 2023-11-24T06:17:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.610+0000 7fc4626ac640 1 -- 172.21.15.5:0/1872222927 learned_addr learned my addr 172.21.15.5:0/1872222927 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.610+0000 7fc461eab640 1 --2- 172.21.15.5:0/1872222927 >> v2:172.21.15.6:3300/0 conn(0x7fc45c0acd70 0x7fc45c06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.610+0000 7fc4626ac640 1 -- 172.21.15.5:0/1872222927 >> v2:172.21.15.6:3300/0 conn(0x7fc45c0acd70 msgr2=0x7fc45c06dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.610+0000 7fc4626ac640 1 --2- 172.21.15.5:0/1872222927 >> v2:172.21.15.6:3300/0 conn(0x7fc45c0acd70 0x7fc45c06dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.610+0000 7fc4626ac640 1 -- 172.21.15.5:0/1872222927 >> v1:172.21.15.6:6789/0 conn(0x7fc45c0ae490 legacy=0x7fc45c0ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.610+0000 7fc4626ac640 1 -- 172.21.15.5:0/1872222927 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c06d150 con 0x7fc45c070490 2023-11-24T06:17:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.611+0000 7fc4626ac640 1 --2- 172.21.15.5:0/1872222927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc45c070490 0x7fc45c070900 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0700f0 tx=0x7fc45402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986d89499cecb3a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.612+0000 7fc460ea9640 1 -- 172.21.15.5:0/1872222927 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45402ca80 con 0x7fc45c070490 2023-11-24T06:17:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.612+0000 7fc460ea9640 1 -- 172.21.15.5:0/1872222927 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45402cbe0 con 0x7fc45c070490 2023-11-24T06:17:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.612+0000 7fc460ea9640 1 -- 172.21.15.5:0/1872222927 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4540343f0 con 0x7fc45c070490 2023-11-24T06:17:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.612+0000 7fc46410f640 1 -- 172.21.15.5:0/1872222927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc45c070490 msgr2=0x7fc45c070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.612+0000 7fc46410f640 1 --2- 172.21.15.5:0/1872222927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc45c070490 0x7fc45c070900 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0700f0 tx=0x7fc45402bee0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.612+0000 7fc46410f640 1 -- 172.21.15.5:0/1872222927 shutdown_connections 2023-11-24T06:17:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.612+0000 7fc46410f640 1 --2- 172.21.15.5:0/1872222927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc45c070490 0x7fc45c070900 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.612+0000 7fc46410f640 1 --2- 172.21.15.5:0/1872222927 >> v2:172.21.15.6:3300/0 conn(0x7fc45c0acd70 0x7fc45c06dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.612+0000 7fc46410f640 1 -- 172.21.15.5:0/1872222927 >> 172.21.15.5:0/1872222927 conn(0x7fc45c0b0ba0 msgr2=0x7fc45c0b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.613+0000 7fc46410f640 1 -- 172.21.15.5:0/1872222927 shutdown_connections 2023-11-24T06:17:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.613+0000 7fc46410f640 1 -- 172.21.15.5:0/1872222927 wait complete. 2023-11-24T06:17:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.614+0000 7fc46410f640 1 Processor -- start 2023-11-24T06:17:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.615+0000 7fc46410f640 1 -- start start 2023-11-24T06:17:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.615+0000 7fc46410f640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc45c070490 0x7fc45c1f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc46410f640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc45c0acd70 0x7fc45c158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc46410f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc45c0ae490 0x7fc45c158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc46410f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c0c9860 con 0x7fc45c0ae490 2023-11-24T06:17:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc46410f640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c1f7760 con 0x7fc45c070490 2023-11-24T06:17:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc46410f640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c0c87e0 con 0x7fc45c0acd70 2023-11-24T06:17:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc4626ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc45c0ae490 0x7fc45c158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc4626ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc45c0ae490 0x7fc45c158c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37082/0 (socket says 172.21.15.5:37082) 2023-11-24T06:17:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc4626ac640 1 -- 172.21.15.5:0/1587773311 learned_addr learned my addr 172.21.15.5:0/1587773311 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc461eab640 1 --2- 172.21.15.5:0/1587773311 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc45c070490 0x7fc45c1f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc4616aa640 1 --2- 172.21.15.5:0/1587773311 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc45c0acd70 0x7fc45c158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc4626ac640 1 -- 172.21.15.5:0/1587773311 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc45c0acd70 msgr2=0x7fc45c158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc4626ac640 1 --2- 172.21.15.5:0/1587773311 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc45c0acd70 0x7fc45c158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc4626ac640 1 -- 172.21.15.5:0/1587773311 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc45c070490 msgr2=0x7fc45c1f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc4626ac640 1 --2- 172.21.15.5:0/1587773311 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc45c070490 0x7fc45c1f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.616+0000 7fc4626ac640 1 -- 172.21.15.5:0/1587773311 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc454009b30 con 0x7fc45c0ae490 2023-11-24T06:17:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.617+0000 7fc4626ac640 1 --2- 172.21.15.5:0/1587773311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc45c0ae490 0x7fc45c158c80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc454036040 tx=0x7fc454004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.617+0000 7fc4527fc640 1 -- 172.21.15.5:0/1587773311 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45403e450 con 0x7fc45c0ae490 2023-11-24T06:17:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.617+0000 7fc4527fc640 1 -- 172.21.15.5:0/1587773311 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45403e5d0 con 0x7fc45c0ae490 2023-11-24T06:17:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.617+0000 7fc4527fc640 1 -- 172.21.15.5:0/1587773311 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45403e980 con 0x7fc45c0ae490 2023-11-24T06:17:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.617+0000 7fc46410f640 1 -- 172.21.15.5:0/1587773311 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45c0c8ac0 con 0x7fc45c0ae490 2023-11-24T06:17:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.617+0000 7fc46410f640 1 -- 172.21.15.5:0/1587773311 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45c0c9080 con 0x7fc45c0ae490 2023-11-24T06:17:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.618+0000 7fc46410f640 1 -- 172.21.15.5:0/1587773311 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45c071880 con 0x7fc45c0ae490 2023-11-24T06:17:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.619+0000 7fc4527fc640 1 -- 172.21.15.5:0/1587773311 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc45402cd50 con 0x7fc45c0ae490 2023-11-24T06:17:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.620+0000 7fc4527fc640 1 --2- 172.21.15.5:0/1587773311 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc44006ae40 0x7fc44006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.621+0000 7fc4527fc640 1 -- 172.21.15.5:0/1587773311 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4540ac1b0 con 0x7fc45c0ae490 2023-11-24T06:17:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.621+0000 7fc461eab640 1 --2- 172.21.15.5:0/1587773311 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc44006ae40 0x7fc44006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.622+0000 7fc461eab640 1 --2- 172.21.15.5:0/1587773311 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc44006ae40 0x7fc44006d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc458006fd0 tx=0x7fc458008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:33.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.626+0000 7fc4527fc640 1 -- 172.21.15.5:0/1587773311 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc4540774c0 con 0x7fc45c0ae490 2023-11-24T06:17:33.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.850+0000 7fc46410f640 1 -- 172.21.15.5:0/1587773311 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol7562", "target": ["mon-mgr", ""]}) v1 -- 0x7fc45c0b3d90 con 0x7fc44006ae40 2023-11-24T06:17:33.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.858+0000 7fc4527fc640 1 -- 172.21.15.5:0/1587773311 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7fc45800c050 con 0x7fc44006ae40 2023-11-24T06:17:33.861 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/_nogroup/subvol7562/e145f047-1c7b-480e-bc3d-2765cfad42ee 2023-11-24T06:17:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.861+0000 7fc46410f640 1 -- 172.21.15.5:0/1587773311 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc44006ae40 msgr2=0x7fc44006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.861+0000 7fc46410f640 1 --2- 172.21.15.5:0/1587773311 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc44006ae40 0x7fc44006d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc458006fd0 tx=0x7fc458008040 comp rx=0 tx=0).stop 2023-11-24T06:17:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.861+0000 7fc46410f640 1 -- 172.21.15.5:0/1587773311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc45c0ae490 msgr2=0x7fc45c158c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.861+0000 7fc46410f640 1 --2- 172.21.15.5:0/1587773311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc45c0ae490 0x7fc45c158c80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc454036040 tx=0x7fc454004a40 comp rx=0 tx=0).stop 2023-11-24T06:17:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.861+0000 7fc46410f640 1 -- 172.21.15.5:0/1587773311 shutdown_connections 2023-11-24T06:17:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.861+0000 7fc46410f640 1 --2- 172.21.15.5:0/1587773311 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc44006ae40 0x7fc44006d300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.861+0000 7fc46410f640 1 --2- 172.21.15.5:0/1587773311 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc45c0acd70 0x7fc45c158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.861+0000 7fc46410f640 1 --2- 172.21.15.5:0/1587773311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc45c0ae490 0x7fc45c158c80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.861+0000 7fc46410f640 1 --2- 172.21.15.5:0/1587773311 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc45c070490 0x7fc45c1f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.861+0000 7fc46410f640 1 -- 172.21.15.5:0/1587773311 >> 172.21.15.5:0/1587773311 conn(0x7fc45c0b0ba0 msgr2=0x7fc45c071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.861+0000 7fc46410f640 1 -- 172.21.15.5:0/1587773311 shutdown_connections 2023-11-24T06:17:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:33.861+0000 7fc46410f640 1 -- 172.21.15.5:0/1587773311 wait complete. 2023-11-24T06:17:33.883 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol7562 with 200 files each 1MB size under directory volumes/_nogroup/subvol7562/e145f047-1c7b-480e-bc3d-2765cfad42ee 2023-11-24T06:17:33.884 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T06:17:33.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:17:34.000 DEBUG:teuthology.orchestra.run.smithi006:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T06:17:34.127 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T06:17:34.127 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:17:34.127 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:17:34.190 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:17:34.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.417+0000 7f050083c640 1 Processor -- start 2023-11-24T06:17:34.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.417+0000 7f050083c640 1 -- start start 2023-11-24T06:17:34.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.419+0000 7f050083c640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f04fc15d370 0x7f04fc0d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.419+0000 7f050083c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f04fc0b1f00 0x7f04fc0cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.419+0000 7f050083c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04fc0d0360 con 0x7f04fc0cff00 2023-11-24T06:17:34.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.419+0000 7f050083c640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04fc0afd30 con 0x7f04fc15d370 2023-11-24T06:17:34.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.419+0000 7f050083c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc0d0c30 con 0x7f04fc0b1f00 2023-11-24T06:17:34.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.419+0000 7f04fb59e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f04fc0cff00 0x7f04fc0cdb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35636/0 (socket says 172.21.15.5:35636) 2023-11-24T06:17:34.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.419+0000 7f04fb59e640 1 -- 172.21.15.5:0/2548999713 learned_addr learned my addr 172.21.15.5:0/2548999713 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:34.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.420+0000 7f04fad9d640 1 --2- 172.21.15.5:0/2548999713 >> v2:172.21.15.6:3300/0 conn(0x7f04fc15d370 0x7f04fc0d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.420+0000 7f04fa59c640 1 --2- 172.21.15.5:0/2548999713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f04fc0b1f00 0x7f04fc0cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.420+0000 7f04f9d9b640 1 -- 172.21.15.5:0/2548999713 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2647919373 0 0) 0x7f04fc0d0360 con 0x7f04fc0cff00 2023-11-24T06:17:34.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.420+0000 7f04f9d9b640 1 -- 172.21.15.5:0/2548999713 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04e4003520 con 0x7f04fc0cff00 2023-11-24T06:17:34.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.420+0000 7f04f9d9b640 1 -- 172.21.15.5:0/2548999713 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3512193897 0 0) 0x7f04e4003520 con 0x7f04fc0cff00 2023-11-24T06:17:34.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.420+0000 7f04f9d9b640 1 -- 172.21.15.5:0/2548999713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f04fc0b1f00 msgr2=0x7f04fc0cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:34.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.420+0000 7f04f9d9b640 1 --2- 172.21.15.5:0/2548999713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f04fc0b1f00 0x7f04fc0cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.420+0000 7f04f9d9b640 1 -- 172.21.15.5:0/2548999713 >> v2:172.21.15.6:3300/0 conn(0x7f04fc15d370 msgr2=0x7f04fc0d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:34.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.420+0000 7f04f9d9b640 1 --2- 172.21.15.5:0/2548999713 >> v2:172.21.15.6:3300/0 conn(0x7f04fc15d370 0x7f04fc0d1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.420+0000 7f04f9d9b640 1 -- 172.21.15.5:0/2548999713 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc0cef60 con 0x7f04fc0cff00 2023-11-24T06:17:34.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.420+0000 7f04f9d9b640 1 -- 172.21.15.5:0/2548999713 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f04f40019f0 con 0x7f04fc0cff00 2023-11-24T06:17:34.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.420+0000 7f04f9d9b640 1 -- 172.21.15.5:0/2548999713 >> v1:172.21.15.5:6789/0 conn(0x7f04fc0cff00 legacy=0x7f04fc0cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:34.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04f9d9b640 1 --2- 172.21.15.5:0/2548999713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40039e0 0x7f04e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04f9d9b640 1 --2- 172.21.15.5:0/2548999713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04e40047c0 0x7f04e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04f9d9b640 1 --2- 172.21.15.5:0/2548999713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f04fc0b1f00 0x7f04e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04f9d9b640 1 -- 172.21.15.5:0/2548999713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc0afd30 con 0x7f04e40039e0 2023-11-24T06:17:34.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04f9d9b640 1 -- 172.21.15.5:0/2548999713 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc0d0c30 con 0x7f04e40047c0 2023-11-24T06:17:34.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04f9d9b640 1 -- 172.21.15.5:0/2548999713 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04e400ba30 con 0x7f04fc0b1f00 2023-11-24T06:17:34.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04fad9d640 1 --2- 172.21.15.5:0/2548999713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40039e0 0x7f04e4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04fb59e640 1 --2- 172.21.15.5:0/2548999713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f04fc0b1f00 0x7f04e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04fa59c640 1 --2- 172.21.15.5:0/2548999713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04e40047c0 0x7f04e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04fad9d640 1 -- 172.21.15.5:0/2548999713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f04fc0b1f00 msgr2=0x7f04e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:34.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04fad9d640 1 --2- 172.21.15.5:0/2548999713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f04fc0b1f00 0x7f04e4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04fad9d640 1 -- 172.21.15.5:0/2548999713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04e40047c0 msgr2=0x7f04e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:34.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04fad9d640 1 --2- 172.21.15.5:0/2548999713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04e40047c0 0x7f04e4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04fad9d640 1 -- 172.21.15.5:0/2548999713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04e4003520 con 0x7f04e40039e0 2023-11-24T06:17:34.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.421+0000 7f04fb59e640 1 --2- 172.21.15.5:0/2548999713 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f04fc0b1f00 0x7f04e4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:34.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.422+0000 7f04fa59c640 1 --2- 172.21.15.5:0/2548999713 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04e40047c0 0x7f04e4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.423+0000 7f04fad9d640 1 --2- 172.21.15.5:0/2548999713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40039e0 0x7f04e4003e00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009a90 tx=0x7f04e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d842c1082c169447 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.423+0000 7f04f9d9b640 1 -- 172.21.15.5:0/2548999713 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec02c6b0 con 0x7f04e40039e0 2023-11-24T06:17:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.423+0000 7f04f9d9b640 1 -- 172.21.15.5:0/2548999713 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ec02c810 con 0x7f04e40039e0 2023-11-24T06:17:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.423+0000 7f04f9d9b640 1 -- 172.21.15.5:0/2548999713 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec02cba0 con 0x7f04e40039e0 2023-11-24T06:17:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.424+0000 7f050083c640 1 -- 172.21.15.5:0/2548999713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40039e0 msgr2=0x7f04e4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:34.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.424+0000 7f050083c640 1 --2- 172.21.15.5:0/2548999713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40039e0 0x7f04e4003e00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009a90 tx=0x7f04e40057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.424+0000 7f04fb59e640 1 -- 172.21.15.5:0/2548999713 reap_dead start 2023-11-24T06:17:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.424+0000 7f050083c640 1 -- 172.21.15.5:0/2548999713 shutdown_connections 2023-11-24T06:17:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.424+0000 7f050083c640 1 -- 172.21.15.5:0/2548999713 >> 172.21.15.5:0/2548999713 conn(0x7f04fc09c3d0 msgr2=0x7f04fc0d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.424+0000 7f050083c640 1 -- 172.21.15.5:0/2548999713 shutdown_connections 2023-11-24T06:17:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.424+0000 7f050083c640 1 -- 172.21.15.5:0/2548999713 wait complete. 2023-11-24T06:17:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.426+0000 7f050083c640 1 Processor -- start 2023-11-24T06:17:34.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.426+0000 7f050083c640 1 -- start start 2023-11-24T06:17:34.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.427+0000 7f050083c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40039e0 0x7f04fc14d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.427+0000 7f050083c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f04fc09ced0 0x7f04fc09d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.427+0000 7f050083c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04fc0caaa0 0x7f04fc0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.427+0000 7f050083c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc09fa50 con 0x7f04e40039e0 2023-11-24T06:17:34.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.427+0000 7f050083c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc09fd30 con 0x7f04fc0caaa0 2023-11-24T06:17:34.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.427+0000 7f050083c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc0a0010 con 0x7f04fc09ced0 2023-11-24T06:17:34.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.428+0000 7f04fad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40039e0 0x7f04fc14d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.428+0000 7f04fad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40039e0 0x7f04fc14d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37102/0 (socket says 172.21.15.5:37102) 2023-11-24T06:17:34.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.428+0000 7f04fad9d640 1 -- 172.21.15.5:0/1799457548 learned_addr learned my addr 172.21.15.5:0/1799457548 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:34.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.428+0000 7f04fa59c640 1 --2- 172.21.15.5:0/1799457548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f04fc09ced0 0x7f04fc09d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.428+0000 7f04fb59e640 1 --2- 172.21.15.5:0/1799457548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04fc0caaa0 0x7f04fc0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.428+0000 7f04fad9d640 1 -- 172.21.15.5:0/1799457548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f04fc09ced0 msgr2=0x7f04fc09d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:34.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.428+0000 7f04fad9d640 1 --2- 172.21.15.5:0/1799457548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f04fc09ced0 0x7f04fc09d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.428+0000 7f04fad9d640 1 -- 172.21.15.5:0/1799457548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04fc0caaa0 msgr2=0x7f04fc0caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:34.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.428+0000 7f04fad9d640 1 --2- 172.21.15.5:0/1799457548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04fc0caaa0 0x7f04fc0caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.428+0000 7f04fad9d640 1 -- 172.21.15.5:0/1799457548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc0a02f0 con 0x7f04e40039e0 2023-11-24T06:17:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.429+0000 7f04fad9d640 1 --2- 172.21.15.5:0/1799457548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40039e0 0x7f04fc14d430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f04ec000c00 tx=0x7f04ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.429+0000 7f04e37fe640 1 -- 172.21.15.5:0/1799457548 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec003810 con 0x7f04e40039e0 2023-11-24T06:17:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.429+0000 7f04e37fe640 1 -- 172.21.15.5:0/1799457548 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ec003990 con 0x7f04e40039e0 2023-11-24T06:17:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.429+0000 7f04e37fe640 1 -- 172.21.15.5:0/1799457548 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec003d40 con 0x7f04e40039e0 2023-11-24T06:17:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.429+0000 7f050083c640 1 -- 172.21.15.5:0/1799457548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04fc0a05d0 con 0x7f04e40039e0 2023-11-24T06:17:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.429+0000 7f050083c640 1 -- 172.21.15.5:0/1799457548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04fc09f670 con 0x7f04e40039e0 2023-11-24T06:17:34.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.431+0000 7f050083c640 1 -- 172.21.15.5:0/1799457548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04fc04e840 con 0x7f04e40039e0 2023-11-24T06:17:34.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.431+0000 7f04e37fe640 1 -- 172.21.15.5:0/1799457548 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f04ec03b070 con 0x7f04e40039e0 2023-11-24T06:17:34.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.432+0000 7f04e37fe640 1 --2- 172.21.15.5:0/1799457548 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f04e806ae90 0x7f04e806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.432+0000 7f04e37fe640 1 -- 172.21.15.5:0/1799457548 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04ec0ab5d0 con 0x7f04e40039e0 2023-11-24T06:17:34.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.433+0000 7f04fa59c640 1 --2- 172.21.15.5:0/1799457548 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f04e806ae90 0x7f04e806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.433+0000 7f04fa59c640 1 --2- 172.21.15.5:0/1799457548 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f04e806ae90 0x7f04e806d350 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0a0bd0 tx=0x7f04f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.437+0000 7f04e37fe640 1 -- 172.21.15.5:0/1799457548 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f04ec076830 con 0x7f04e40039e0 2023-11-24T06:17:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.630+0000 7f050083c640 1 -- 172.21.15.5:0/1799457548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04fc0d14f0 con 0x7f04e40039e0 2023-11-24T06:17:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.632+0000 7f04e37fe640 1 -- 172.21.15.5:0/1799457548 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16425 (secure 0 0 0) 0x7f04ec079e50 con 0x7f04e40039e0 2023-11-24T06:17:34.636 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:34.636 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:17:18.380558+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:17:14.868917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:17:15.895189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:17:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.635+0000 7f050083c640 1 -- 172.21.15.5:0/1799457548 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f04e806ae90 msgr2=0x7f04e806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.635+0000 7f050083c640 1 --2- 172.21.15.5:0/1799457548 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f04e806ae90 0x7f04e806d350 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f04fc0a0bd0 tx=0x7f04f0008040 comp rx=0 tx=0).stop 2023-11-24T06:17:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.635+0000 7f050083c640 1 -- 172.21.15.5:0/1799457548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40039e0 msgr2=0x7f04fc14d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.635+0000 7f050083c640 1 --2- 172.21.15.5:0/1799457548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40039e0 0x7f04fc14d430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f04ec000c00 tx=0x7f04ec004730 comp rx=0 tx=0).stop 2023-11-24T06:17:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.635+0000 7f050083c640 1 -- 172.21.15.5:0/1799457548 shutdown_connections 2023-11-24T06:17:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.635+0000 7f050083c640 1 --2- 172.21.15.5:0/1799457548 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f04e806ae90 0x7f04e806d350 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.635+0000 7f050083c640 1 --2- 172.21.15.5:0/1799457548 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f04fc09ced0 0x7f04fc09d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.635+0000 7f050083c640 1 --2- 172.21.15.5:0/1799457548 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f04fc0caaa0 0x7f04fc0caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.635+0000 7f050083c640 1 --2- 172.21.15.5:0/1799457548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e40039e0 0x7f04fc14d430 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.635+0000 7f050083c640 1 -- 172.21.15.5:0/1799457548 >> 172.21.15.5:0/1799457548 conn(0x7f04fc09c3d0 msgr2=0x7f04fc0ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.635+0000 7f050083c640 1 -- 172.21.15.5:0/1799457548 shutdown_connections 2023-11-24T06:17:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.636+0000 7f050083c640 1 -- 172.21.15.5:0/1799457548 wait complete. 2023-11-24T06:17:34.657 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:17:14.868917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:17:15.895189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:17:34.657 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:17:34.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.876+0000 7fdf4a7ac640 1 Processor -- start 2023-11-24T06:17:34.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.877+0000 7fdf4a7ac640 1 -- start start 2023-11-24T06:17:34.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.878+0000 7fdf4a7ac640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fdf440ad450 0x7fdf44154280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.878+0000 7fdf4a7ac640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf44070850 0x7fdf44071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.878+0000 7fdf4a7ac640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf440ce650 con 0x7fdf44153ea0 2023-11-24T06:17:34.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.878+0000 7fdf4a7ac640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf440ad8e0 con 0x7fdf440ad450 2023-11-24T06:17:34.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.878+0000 7fdf4a7ac640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf441547c0 con 0x7fdf44070850 2023-11-24T06:17:34.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.879+0000 7fdf497aa640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdf44153ea0 0x7fdf4415d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35650/0 (socket says 172.21.15.5:35650) 2023-11-24T06:17:34.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.879+0000 7fdf497aa640 1 -- 172.21.15.5:0/3738525214 learned_addr learned my addr 172.21.15.5:0/3738525214 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:34.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.879+0000 7fdf49fab640 1 --2- 172.21.15.5:0/3738525214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf44070850 0x7fdf44071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.879+0000 7fdf48fa9640 1 --2- 172.21.15.5:0/3738525214 >> v2:172.21.15.6:3300/0 conn(0x7fdf440ad450 0x7fdf44154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.880+0000 7fdf3bfff640 1 -- 172.21.15.5:0/3738525214 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064869717 0 0) 0x7fdf440ce650 con 0x7fdf44153ea0 2023-11-24T06:17:34.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.880+0000 7fdf3bfff640 1 -- 172.21.15.5:0/3738525214 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf30003520 con 0x7fdf44153ea0 2023-11-24T06:17:34.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.880+0000 7fdf49fab640 1 -- 172.21.15.5:0/3738525214 >> v2:172.21.15.6:3300/0 conn(0x7fdf440ad450 msgr2=0x7fdf44154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:34.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.880+0000 7fdf49fab640 1 --2- 172.21.15.5:0/3738525214 >> v2:172.21.15.6:3300/0 conn(0x7fdf440ad450 0x7fdf44154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.880+0000 7fdf49fab640 1 -- 172.21.15.5:0/3738525214 >> v1:172.21.15.5:6789/0 conn(0x7fdf44153ea0 legacy=0x7fdf4415d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:34.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.880+0000 7fdf49fab640 1 -- 172.21.15.5:0/3738525214 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4415e0a0 con 0x7fdf44070850 2023-11-24T06:17:34.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.880+0000 7fdf48fa9640 1 --2- 172.21.15.5:0/3738525214 >> v2:172.21.15.6:3300/0 conn(0x7fdf440ad450 0x7fdf44154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T06:17:34.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.881+0000 7fdf49fab640 1 --2- 172.21.15.5:0/3738525214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf44070850 0x7fdf44071350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009d90 tx=0x7fdf3c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91096568bb9e2a05 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:34.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.882+0000 7fdf3bfff640 1 -- 172.21.15.5:0/3738525214 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c02cc50 con 0x7fdf44070850 2023-11-24T06:17:34.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.882+0000 7fdf3bfff640 1 -- 172.21.15.5:0/3738525214 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3c02cdb0 con 0x7fdf44070850 2023-11-24T06:17:34.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.882+0000 7fdf3bfff640 1 -- 172.21.15.5:0/3738525214 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c034640 con 0x7fdf44070850 2023-11-24T06:17:34.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.883+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3738525214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf44070850 msgr2=0x7fdf44071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:34.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.883+0000 7fdf4a7ac640 1 --2- 172.21.15.5:0/3738525214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf44070850 0x7fdf44071350 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009d90 tx=0x7fdf3c02bea0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.883+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3738525214 shutdown_connections 2023-11-24T06:17:34.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.883+0000 7fdf4a7ac640 1 --2- 172.21.15.5:0/3738525214 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf44070850 0x7fdf44071350 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.883+0000 7fdf4a7ac640 1 --2- 172.21.15.5:0/3738525214 >> v2:172.21.15.6:3300/0 conn(0x7fdf440ad450 0x7fdf44154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.883+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3738525214 >> 172.21.15.5:0/3738525214 conn(0x7fdf440cec10 msgr2=0x7fdf440cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:34.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.883+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3738525214 shutdown_connections 2023-11-24T06:17:34.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.883+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3738525214 wait complete. 2023-11-24T06:17:34.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.885+0000 7fdf4a7ac640 1 Processor -- start 2023-11-24T06:17:34.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.885+0000 7fdf4a7ac640 1 -- start start 2023-11-24T06:17:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.886+0000 7fdf4a7ac640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf44070850 0x7fdf440aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.886+0000 7fdf4a7ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf440ad450 0x7fdf4406d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.886+0000 7fdf4a7ac640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf44153ea0 0x7fdf4406d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.886+0000 7fdf4a7ac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf440b01a0 con 0x7fdf440ad450 2023-11-24T06:17:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.886+0000 7fdf4a7ac640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf440af370 con 0x7fdf44153ea0 2023-11-24T06:17:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.886+0000 7fdf4a7ac640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf440af650 con 0x7fdf44070850 2023-11-24T06:17:34.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.886+0000 7fdf48fa9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf440ad450 0x7fdf4406d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.886+0000 7fdf48fa9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf440ad450 0x7fdf4406d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37110/0 (socket says 172.21.15.5:37110) 2023-11-24T06:17:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.886+0000 7fdf48fa9640 1 -- 172.21.15.5:0/3847134672 learned_addr learned my addr 172.21.15.5:0/3847134672 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.887+0000 7fdf49fab640 1 --2- 172.21.15.5:0/3847134672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf44153ea0 0x7fdf4406d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.887+0000 7fdf497aa640 1 --2- 172.21.15.5:0/3847134672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf44070850 0x7fdf440aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.887+0000 7fdf48fa9640 1 -- 172.21.15.5:0/3847134672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf44070850 msgr2=0x7fdf440aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.887+0000 7fdf48fa9640 1 --2- 172.21.15.5:0/3847134672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf44070850 0x7fdf440aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.887+0000 7fdf48fa9640 1 -- 172.21.15.5:0/3847134672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf44153ea0 msgr2=0x7fdf4406d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.887+0000 7fdf48fa9640 1 --2- 172.21.15.5:0/3847134672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf44153ea0 0x7fdf4406d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.887+0000 7fdf48fa9640 1 -- 172.21.15.5:0/3847134672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34009400 con 0x7fdf440ad450 2023-11-24T06:17:34.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.887+0000 7fdf48fa9640 1 --2- 172.21.15.5:0/3847134672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf440ad450 0x7fdf4406d2c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400ebe0 tx=0x7fdf3400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:34.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.888+0000 7fdf39ffb640 1 -- 172.21.15.5:0/3847134672 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf34009760 con 0x7fdf440ad450 2023-11-24T06:17:34.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.888+0000 7fdf39ffb640 1 -- 172.21.15.5:0/3847134672 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3400cc30 con 0x7fdf440ad450 2023-11-24T06:17:34.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.888+0000 7fdf39ffb640 1 -- 172.21.15.5:0/3847134672 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3400eee0 con 0x7fdf440ad450 2023-11-24T06:17:34.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.888+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3847134672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c009a70 con 0x7fdf440ad450 2023-11-24T06:17:34.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.888+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3847134672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf440afc10 con 0x7fdf440ad450 2023-11-24T06:17:34.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.889+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3847134672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4404e840 con 0x7fdf440ad450 2023-11-24T06:17:34.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.890+0000 7fdf39ffb640 1 -- 172.21.15.5:0/3847134672 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdf34013470 con 0x7fdf440ad450 2023-11-24T06:17:34.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.892+0000 7fdf39ffb640 1 --2- 172.21.15.5:0/3847134672 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf2806ae70 0x7fdf2806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:34.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.892+0000 7fdf39ffb640 1 -- 172.21.15.5:0/3847134672 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf34089920 con 0x7fdf440ad450 2023-11-24T06:17:34.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.892+0000 7fdf497aa640 1 --2- 172.21.15.5:0/3847134672 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf2806ae70 0x7fdf2806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:34.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.893+0000 7fdf497aa640 1 --2- 172.21.15.5:0/3847134672 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf2806ae70 0x7fdf2806d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf400011e0 tx=0x7fdf40009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:34.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.897+0000 7fdf39ffb640 1 -- 172.21.15.5:0/3847134672 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf340549c0 con 0x7fdf440ad450 2023-11-24T06:17:34.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:34.908+0000 7fdf39ffb640 1 -- 172.21.15.5:0/3847134672 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fdf34054d90 con 0x7fdf440ad450 2023-11-24T06:17:35.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.095+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3847134672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf440aeae0 con 0x7fdf440ad450 2023-11-24T06:17:35.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.096+0000 7fdf39ffb640 1 -- 172.21.15.5:0/3847134672 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf34087730 con 0x7fdf440ad450 2023-11-24T06:17:35.099 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:17:35.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.099+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3847134672 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf2806ae70 msgr2=0x7fdf2806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:35.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.099+0000 7fdf4a7ac640 1 --2- 172.21.15.5:0/3847134672 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf2806ae70 0x7fdf2806d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdf400011e0 tx=0x7fdf40009040 comp rx=0 tx=0).stop 2023-11-24T06:17:35.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.099+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3847134672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf440ad450 msgr2=0x7fdf4406d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:35.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.099+0000 7fdf4a7ac640 1 --2- 172.21.15.5:0/3847134672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf440ad450 0x7fdf4406d2c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400ebe0 tx=0x7fdf3400c520 comp rx=0 tx=0).stop 2023-11-24T06:17:35.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.099+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3847134672 shutdown_connections 2023-11-24T06:17:35.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.099+0000 7fdf4a7ac640 1 --2- 172.21.15.5:0/3847134672 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf2806ae70 0x7fdf2806d330 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.099+0000 7fdf4a7ac640 1 --2- 172.21.15.5:0/3847134672 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf44153ea0 0x7fdf4406d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.099+0000 7fdf4a7ac640 1 --2- 172.21.15.5:0/3847134672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf440ad450 0x7fdf4406d2c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.099+0000 7fdf4a7ac640 1 --2- 172.21.15.5:0/3847134672 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf44070850 0x7fdf440aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.099+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3847134672 >> 172.21.15.5:0/3847134672 conn(0x7fdf440cec10 msgr2=0x7fdf440ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:35.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.099+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3847134672 shutdown_connections 2023-11-24T06:17:35.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.099+0000 7fdf4a7ac640 1 -- 172.21.15.5:0/3847134672 wait complete. 2023-11-24T06:17:35.122 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:17:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.350+0000 7f1a65f1d640 1 Processor -- start 2023-11-24T06:17:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.351+0000 7f1a65f1d640 1 -- start start 2023-11-24T06:17:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.352+0000 7f1a65f1d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015b350 0x7f1a600abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.352+0000 7f1a65f1d640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1a6009cb30 0x7f1a600b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.353+0000 7f1a65f1d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6015b730 con 0x7f1a600adb60 2023-11-24T06:17:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.353+0000 7f1a65f1d640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a600ac740 con 0x7f1a6009cb30 2023-11-24T06:17:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.353+0000 7f1a65f1d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a600ac460 con 0x7f1a6015b350 2023-11-24T06:17:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.353+0000 7f1a5f7fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015b350 0x7f1a600abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.353+0000 7f1a5f7fe640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015b350 0x7f1a600abe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:60524/0 (socket says 172.21.15.5:60524) 2023-11-24T06:17:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.353+0000 7f1a5f7fe640 1 -- 172.21.15.5:0/500272414 learned_addr learned my addr 172.21.15.5:0/500272414 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.353+0000 7f1a5effd640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f1a6009cb30 0x7f1a600b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.354+0000 7f1a5f7fe640 1 -- 172.21.15.5:0/500272414 >> v2:172.21.15.6:3300/0 conn(0x7f1a6009cb30 msgr2=0x7f1a600b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.354+0000 7f1a5f7fe640 1 --2- 172.21.15.5:0/500272414 >> v2:172.21.15.6:3300/0 conn(0x7f1a6009cb30 0x7f1a600b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.354+0000 7f1a5f7fe640 1 -- 172.21.15.5:0/500272414 >> v1:172.21.15.6:6789/0 conn(0x7f1a600adb60 legacy=0x7f1a600adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:35.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.354+0000 7f1a5f7fe640 1 -- 172.21.15.5:0/500272414 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a600b2a50 con 0x7f1a6015b350 2023-11-24T06:17:35.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.356+0000 7f1a5f7fe640 1 --2- 172.21.15.5:0/500272414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015b350 0x7f1a600abe80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009e50 tx=0x7f1a5002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79a685e4e0a8a839 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.356+0000 7f1a5e7fc640 1 -- 172.21.15.5:0/500272414 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5002cb00 con 0x7f1a6015b350 2023-11-24T06:17:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.356+0000 7f1a5e7fc640 1 -- 172.21.15.5:0/500272414 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5002cc60 con 0x7f1a6015b350 2023-11-24T06:17:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.356+0000 7f1a5e7fc640 1 -- 172.21.15.5:0/500272414 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a500343f0 con 0x7f1a6015b350 2023-11-24T06:17:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.357+0000 7f1a65f1d640 1 -- 172.21.15.5:0/500272414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015b350 msgr2=0x7f1a600abe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.357+0000 7f1a65f1d640 1 --2- 172.21.15.5:0/500272414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015b350 0x7f1a600abe80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009e50 tx=0x7f1a5002bf60 comp rx=0 tx=0).stop 2023-11-24T06:17:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.357+0000 7f1a65f1d640 1 -- 172.21.15.5:0/500272414 shutdown_connections 2023-11-24T06:17:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.357+0000 7f1a65f1d640 1 --2- 172.21.15.5:0/500272414 >> v2:172.21.15.6:3300/0 conn(0x7f1a6009cb30 0x7f1a600b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.357+0000 7f1a65f1d640 1 --2- 172.21.15.5:0/500272414 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015b350 0x7f1a600abe80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.357+0000 7f1a65f1d640 1 -- 172.21.15.5:0/500272414 >> 172.21.15.5:0/500272414 conn(0x7f1a600b0ba0 msgr2=0x7f1a600b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.357+0000 7f1a65f1d640 1 -- 172.21.15.5:0/500272414 shutdown_connections 2023-11-24T06:17:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.357+0000 7f1a65f1d640 1 -- 172.21.15.5:0/500272414 wait complete. 2023-11-24T06:17:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.359+0000 7f1a65f1d640 1 Processor -- start 2023-11-24T06:17:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.359+0000 7f1a65f1d640 1 -- start start 2023-11-24T06:17:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.360+0000 7f1a65f1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a6009cb30 0x7f1a6013ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.360+0000 7f1a65f1d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a600adb60 0x7f1a6014e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.360+0000 7f1a65f1d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015b350 0x7f1a60159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.360+0000 7f1a65f1d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6014c9e0 con 0x7f1a6009cb30 2023-11-24T06:17:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.360+0000 7f1a65f1d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6013ef80 con 0x7f1a600adb60 2023-11-24T06:17:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.360+0000 7f1a65f1d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6014bdf0 con 0x7f1a6015b350 2023-11-24T06:17:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.360+0000 7f1a5f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a6009cb30 0x7f1a6013ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.361+0000 7f1a5f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a6009cb30 0x7f1a6013ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37124/0 (socket says 172.21.15.5:37124) 2023-11-24T06:17:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.361+0000 7f1a5f7fe640 1 -- 172.21.15.5:0/2264621179 learned_addr learned my addr 172.21.15.5:0/2264621179 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.361+0000 7f1a5ffff640 1 --2- 172.21.15.5:0/2264621179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015b350 0x7f1a60159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.361+0000 7f1a5effd640 1 --2- 172.21.15.5:0/2264621179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a600adb60 0x7f1a6014e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.361+0000 7f1a5f7fe640 1 -- 172.21.15.5:0/2264621179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015b350 msgr2=0x7f1a60159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.361+0000 7f1a5f7fe640 1 --2- 172.21.15.5:0/2264621179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015b350 0x7f1a60159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.361+0000 7f1a5f7fe640 1 -- 172.21.15.5:0/2264621179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a600adb60 msgr2=0x7f1a6014e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.361+0000 7f1a5f7fe640 1 --2- 172.21.15.5:0/2264621179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a600adb60 0x7f1a6014e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.361+0000 7f1a5f7fe640 1 -- 172.21.15.5:0/2264621179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a50009b30 con 0x7f1a6009cb30 2023-11-24T06:17:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.362+0000 7f1a5f7fe640 1 --2- 172.21.15.5:0/2264621179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a6009cb30 0x7f1a6013ea40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1a50000f80 tx=0x7f1a50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.362+0000 7f1a57fff640 1 -- 172.21.15.5:0/2264621179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50009510 con 0x7f1a6009cb30 2023-11-24T06:17:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.362+0000 7f1a57fff640 1 -- 172.21.15.5:0/2264621179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5003e450 con 0x7f1a6009cb30 2023-11-24T06:17:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.362+0000 7f1a57fff640 1 -- 172.21.15.5:0/2264621179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5003e800 con 0x7f1a6009cb30 2023-11-24T06:17:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.362+0000 7f1a65f1d640 1 -- 172.21.15.5:0/2264621179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6014c0b0 con 0x7f1a6009cb30 2023-11-24T06:17:35.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.362+0000 7f1a65f1d640 1 -- 172.21.15.5:0/2264621179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6014c670 con 0x7f1a6009cb30 2023-11-24T06:17:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.363+0000 7f1a65f1d640 1 -- 172.21.15.5:0/2264621179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6014c800 con 0x7f1a6009cb30 2023-11-24T06:17:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.364+0000 7f1a57fff640 1 -- 172.21.15.5:0/2264621179 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1a5003e960 con 0x7f1a6009cb30 2023-11-24T06:17:35.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.365+0000 7f1a57fff640 1 --2- 172.21.15.5:0/2264621179 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1a4006af00 0x7f1a4006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:35.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.365+0000 7f1a57fff640 1 -- 172.21.15.5:0/2264621179 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a500ac420 con 0x7f1a6009cb30 2023-11-24T06:17:35.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.365+0000 7f1a5effd640 1 --2- 172.21.15.5:0/2264621179 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1a4006af00 0x7f1a4006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:35.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.366+0000 7f1a5effd640 1 --2- 172.21.15.5:0/2264621179 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1a4006af00 0x7f1a4006d3c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1a6006e450 tx=0x7f1a48008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:35.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.370+0000 7f1a57fff640 1 -- 172.21.15.5:0/2264621179 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1a500773f0 con 0x7f1a6009cb30 2023-11-24T06:17:35.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.570+0000 7f1a65f1d640 1 -- 172.21.15.5:0/2264621179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a6014d5d0 con 0x7f1a6009cb30 2023-11-24T06:17:35.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.571+0000 7f1a57fff640 1 -- 172.21.15.5:0/2264621179 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1a5007aa10 con 0x7f1a6009cb30 2023-11-24T06:17:35.574 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:17:35.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.573+0000 7f1a65f1d640 1 -- 172.21.15.5:0/2264621179 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1a4006af00 msgr2=0x7f1a4006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:35.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.574+0000 7f1a65f1d640 1 --2- 172.21.15.5:0/2264621179 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1a4006af00 0x7f1a4006d3c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1a6006e450 tx=0x7f1a48008040 comp rx=0 tx=0).stop 2023-11-24T06:17:35.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.574+0000 7f1a65f1d640 1 -- 172.21.15.5:0/2264621179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a6009cb30 msgr2=0x7f1a6013ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:35.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.574+0000 7f1a65f1d640 1 --2- 172.21.15.5:0/2264621179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a6009cb30 0x7f1a6013ea40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1a50000f80 tx=0x7f1a50004a40 comp rx=0 tx=0).stop 2023-11-24T06:17:35.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.574+0000 7f1a65f1d640 1 -- 172.21.15.5:0/2264621179 shutdown_connections 2023-11-24T06:17:35.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.574+0000 7f1a65f1d640 1 --2- 172.21.15.5:0/2264621179 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f1a4006af00 0x7f1a4006d3c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.574+0000 7f1a65f1d640 1 --2- 172.21.15.5:0/2264621179 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f1a6015b350 0x7f1a60159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.574+0000 7f1a65f1d640 1 --2- 172.21.15.5:0/2264621179 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f1a600adb60 0x7f1a6014e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.574+0000 7f1a65f1d640 1 --2- 172.21.15.5:0/2264621179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a6009cb30 0x7f1a6013ea40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.574+0000 7f1a65f1d640 1 -- 172.21.15.5:0/2264621179 >> 172.21.15.5:0/2264621179 conn(0x7f1a600b0ba0 msgr2=0x7f1a6006db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:35.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.574+0000 7f1a65f1d640 1 -- 172.21.15.5:0/2264621179 shutdown_connections 2023-11-24T06:17:35.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.574+0000 7f1a65f1d640 1 -- 172.21.15.5:0/2264621179 wait complete. 2023-11-24T06:17:35.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:17:35.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.818+0000 7fa37751d640 1 Processor -- start 2023-11-24T06:17:35.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.818+0000 7fa37751d640 1 -- start start 2023-11-24T06:17:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.820+0000 7fa37751d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa370162220 0x7fa3700b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.820+0000 7fa37751d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa37006cad0 0x7fa370162a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.820+0000 7fa37751d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa370162fa0 con 0x7fa370162600 2023-11-24T06:17:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.820+0000 7fa37751d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3700706a0 con 0x7fa37006cad0 2023-11-24T06:17:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.820+0000 7fa37751d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa370070c60 con 0x7fa370162220 2023-11-24T06:17:35.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.820+0000 7fa374ab8640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa370162600 0x7fa37006c3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35654/0 (socket says 172.21.15.5:35654) 2023-11-24T06:17:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.820+0000 7fa374ab8640 1 -- 172.21.15.5:0/1252092493 learned_addr learned my addr 172.21.15.5:0/1252092493 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.820+0000 7fa375aba640 1 --2- 172.21.15.5:0/1252092493 >> v2:172.21.15.5:3300/0 conn(0x7fa37006cad0 0x7fa370162a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.821+0000 7fa3752b9640 1 --2- 172.21.15.5:0/1252092493 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa370162220 0x7fa3700b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.821+0000 7fa367fff640 1 -- 172.21.15.5:0/1252092493 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962871817 0 0) 0x7fa370162fa0 con 0x7fa370162600 2023-11-24T06:17:35.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.821+0000 7fa367fff640 1 -- 172.21.15.5:0/1252092493 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa35c003520 con 0x7fa370162600 2023-11-24T06:17:35.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.821+0000 7fa375aba640 1 -- 172.21.15.5:0/1252092493 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa370162220 msgr2=0x7fa3700b1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:35.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.821+0000 7fa375aba640 1 --2- 172.21.15.5:0/1252092493 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa370162220 0x7fa3700b1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.821+0000 7fa375aba640 1 -- 172.21.15.5:0/1252092493 >> v1:172.21.15.5:6789/0 conn(0x7fa370162600 legacy=0x7fa37006c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:35.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.821+0000 7fa375aba640 1 -- 172.21.15.5:0/1252092493 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3700722b0 con 0x7fa37006cad0 2023-11-24T06:17:35.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.822+0000 7fa375aba640 1 --2- 172.21.15.5:0/1252092493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37006cad0 0x7fa370162a60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa368009d90 tx=0x7fa36802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f0f276d1ba656f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:35.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.823+0000 7fa367fff640 1 -- 172.21.15.5:0/1252092493 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36802cc50 con 0x7fa37006cad0 2023-11-24T06:17:35.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.823+0000 7fa367fff640 1 -- 172.21.15.5:0/1252092493 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36802cdb0 con 0x7fa37006cad0 2023-11-24T06:17:35.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.823+0000 7fa367fff640 1 -- 172.21.15.5:0/1252092493 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368034640 con 0x7fa37006cad0 2023-11-24T06:17:35.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.823+0000 7fa37751d640 1 -- 172.21.15.5:0/1252092493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37006cad0 msgr2=0x7fa370162a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.823+0000 7fa37751d640 1 --2- 172.21.15.5:0/1252092493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37006cad0 0x7fa370162a60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa368009d90 tx=0x7fa36802bea0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.823+0000 7fa37751d640 1 -- 172.21.15.5:0/1252092493 shutdown_connections 2023-11-24T06:17:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.823+0000 7fa37751d640 1 --2- 172.21.15.5:0/1252092493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa37006cad0 0x7fa370162a60 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.823+0000 7fa37751d640 1 --2- 172.21.15.5:0/1252092493 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa370162220 0x7fa3700b1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.823+0000 7fa37751d640 1 -- 172.21.15.5:0/1252092493 >> 172.21.15.5:0/1252092493 conn(0x7fa3700b0ba0 msgr2=0x7fa3700b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.823+0000 7fa37751d640 1 -- 172.21.15.5:0/1252092493 shutdown_connections 2023-11-24T06:17:35.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.823+0000 7fa37751d640 1 -- 172.21.15.5:0/1252092493 wait complete. 2023-11-24T06:17:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.825+0000 7fa37751d640 1 Processor -- start 2023-11-24T06:17:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.825+0000 7fa37751d640 1 -- start start 2023-11-24T06:17:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.826+0000 7fa37751d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa37006cad0 0x7fa370153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.826+0000 7fa37751d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa370162220 0x7fa3700afef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.826+0000 7fa37751d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa370162600 0x7fa3700add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.826+0000 7fa37751d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa370155c80 con 0x7fa370162220 2023-11-24T06:17:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.826+0000 7fa37751d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa370155000 con 0x7fa37006cad0 2023-11-24T06:17:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.826+0000 7fa37751d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3701552e0 con 0x7fa370162600 2023-11-24T06:17:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.827+0000 7fa374ab8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa370162220 0x7fa3700afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.827+0000 7fa374ab8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa370162220 0x7fa3700afef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37142/0 (socket says 172.21.15.5:37142) 2023-11-24T06:17:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.827+0000 7fa374ab8640 1 -- 172.21.15.5:0/2319361243 learned_addr learned my addr 172.21.15.5:0/2319361243 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.827+0000 7fa375aba640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa370162600 0x7fa3700add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.827+0000 7fa3752b9640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa37006cad0 0x7fa370153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.827+0000 7fa374ab8640 1 -- 172.21.15.5:0/2319361243 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa370162600 msgr2=0x7fa3700add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.827+0000 7fa374ab8640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa370162600 0x7fa3700add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.827+0000 7fa374ab8640 1 -- 172.21.15.5:0/2319361243 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa37006cad0 msgr2=0x7fa370153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.827+0000 7fa374ab8640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa37006cad0 0x7fa370153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.827+0000 7fa374ab8640 1 -- 172.21.15.5:0/2319361243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa368009a70 con 0x7fa370162220 2023-11-24T06:17:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.827+0000 7fa375aba640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa370162600 0x7fa3700add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.828+0000 7fa374ab8640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa370162220 0x7fa3700afef0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa36000c430 tx=0x7fa36000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.828+0000 7fa365ffb640 1 -- 172.21.15.5:0/2319361243 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360005ce0 con 0x7fa370162220 2023-11-24T06:17:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.828+0000 7fa365ffb640 1 -- 172.21.15.5:0/2319361243 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa360005e40 con 0x7fa370162220 2023-11-24T06:17:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.828+0000 7fa365ffb640 1 -- 172.21.15.5:0/2319361243 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3600072f0 con 0x7fa370162220 2023-11-24T06:17:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.828+0000 7fa37751d640 1 -- 172.21.15.5:0/2319361243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3701555c0 con 0x7fa370162220 2023-11-24T06:17:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.828+0000 7fa37751d640 1 -- 172.21.15.5:0/2319361243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3701560a0 con 0x7fa370162220 2023-11-24T06:17:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.829+0000 7fa37751d640 1 -- 172.21.15.5:0/2319361243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37009d200 con 0x7fa370162220 2023-11-24T06:17:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.830+0000 7fa365ffb640 1 -- 172.21.15.5:0/2319361243 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fa360007450 con 0x7fa370162220 2023-11-24T06:17:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.831+0000 7fa365ffb640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa35406aed0 0x7fa35406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.831+0000 7fa365ffb640 1 -- 172.21.15.5:0/2319361243 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa360088e10 con 0x7fa370162220 2023-11-24T06:17:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.831+0000 7fa3752b9640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa35406aed0 0x7fa35406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:35.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.832+0000 7fa3752b9640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa35406aed0 0x7fa35406d390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa36c006fd0 tx=0x7fa36c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:35.836+0000 7fa365ffb640 1 -- 172.21.15.5:0/2319361243 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa36008c030 con 0x7fa370162220 2023-11-24T06:17:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.033+0000 7fa37751d640 1 -- 172.21.15.5:0/2319361243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa3701538a0 con 0x7fa370162220 2023-11-24T06:17:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.034+0000 7fa365ffb640 1 -- 172.21.15.5:0/2319361243 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa360017020 con 0x7fa370162220 2023-11-24T06:17:36.037 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:17:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.037+0000 7fa37751d640 1 -- 172.21.15.5:0/2319361243 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa35406aed0 msgr2=0x7fa35406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.037+0000 7fa37751d640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa35406aed0 0x7fa35406d390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa36c006fd0 tx=0x7fa36c008040 comp rx=0 tx=0).stop 2023-11-24T06:17:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.037+0000 7fa37751d640 1 -- 172.21.15.5:0/2319361243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa370162220 msgr2=0x7fa3700afef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.037+0000 7fa37751d640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa370162220 0x7fa3700afef0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa36000c430 tx=0x7fa36000c8f0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.037+0000 7fa37751d640 1 -- 172.21.15.5:0/2319361243 shutdown_connections 2023-11-24T06:17:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.037+0000 7fa37751d640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa35406aed0 0x7fa35406d390 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.037+0000 7fa37751d640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa370162600 0x7fa3700add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.037+0000 7fa37751d640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa370162220 0x7fa3700afef0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.037+0000 7fa37751d640 1 --2- 172.21.15.5:0/2319361243 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa37006cad0 0x7fa370153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.037+0000 7fa37751d640 1 -- 172.21.15.5:0/2319361243 >> 172.21.15.5:0/2319361243 conn(0x7fa3700b0ba0 msgr2=0x7fa370161290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.038+0000 7fa37751d640 1 -- 172.21.15.5:0/2319361243 shutdown_connections 2023-11-24T06:17:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.038+0000 7fa37751d640 1 -- 172.21.15.5:0/2319361243 wait complete. 2023-11-24T06:17:36.061 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:17:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.286+0000 7ff578f67640 1 Processor -- start 2023-11-24T06:17:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.286+0000 7ff578f67640 1 -- start start 2023-11-24T06:17:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.287+0000 7ff578f67640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff5740cf710 0x7ff5740cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.287+0000 7ff578f67640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5740d1670 0x7ff5740d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.287+0000 7ff578f67640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5740d04b0 con 0x7ff57415d5d0 2023-11-24T06:17:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.288+0000 7ff578f67640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5740ce4e0 con 0x7ff5740cf710 2023-11-24T06:17:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.288+0000 7ff578f67640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740cf010 con 0x7ff5740d1670 2023-11-24T06:17:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.288+0000 7ff5737fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff57415d5d0 0x7ff5740ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35658/0 (socket says 172.21.15.5:35658) 2023-11-24T06:17:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.288+0000 7ff5737fe640 1 -- 172.21.15.5:0/1007679565 learned_addr learned my addr 172.21.15.5:0/1007679565 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.288+0000 7ff56affd640 1 --2- 172.21.15.5:0/1007679565 >> v2:172.21.15.6:3300/0 conn(0x7ff5740cf710 0x7ff5740cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.288+0000 7ff573fff640 1 --2- 172.21.15.5:0/1007679565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5740d1670 0x7ff5740d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.288+0000 7ff572ffd640 1 -- 172.21.15.5:0/1007679565 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044208562 0 0) 0x7ff5740d04b0 con 0x7ff57415d5d0 2023-11-24T06:17:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.288+0000 7ff572ffd640 1 -- 172.21.15.5:0/1007679565 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff560003520 con 0x7ff57415d5d0 2023-11-24T06:17:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff572ffd640 1 -- 172.21.15.5:0/1007679565 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1403560105 0 0) 0x7ff560003520 con 0x7ff57415d5d0 2023-11-24T06:17:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff572ffd640 1 -- 172.21.15.5:0/1007679565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5740d1670 msgr2=0x7ff5740d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff572ffd640 1 --2- 172.21.15.5:0/1007679565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5740d1670 0x7ff5740d1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff572ffd640 1 -- 172.21.15.5:0/1007679565 >> v2:172.21.15.6:3300/0 conn(0x7ff5740cf710 msgr2=0x7ff5740cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff572ffd640 1 --2- 172.21.15.5:0/1007679565 >> v2:172.21.15.6:3300/0 conn(0x7ff5740cf710 0x7ff5740cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff572ffd640 1 -- 172.21.15.5:0/1007679565 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5740cdb80 con 0x7ff57415d5d0 2023-11-24T06:17:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff572ffd640 1 -- 172.21.15.5:0/1007679565 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7ff5640019f0 con 0x7ff57415d5d0 2023-11-24T06:17:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff572ffd640 1 -- 172.21.15.5:0/1007679565 >> v1:172.21.15.5:6789/0 conn(0x7ff57415d5d0 legacy=0x7ff5740ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff572ffd640 1 --2- 172.21.15.5:0/1007679565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5740d1670 0x7ff5600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff572ffd640 1 --2- 172.21.15.5:0/1007679565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff560003fb0 0x7ff5600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff572ffd640 1 --2- 172.21.15.5:0/1007679565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff560004d00 0x7ff560005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff572ffd640 1 -- 172.21.15.5:0/1007679565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740ce4e0 con 0x7ff560004d00 2023-11-24T06:17:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff572ffd640 1 -- 172.21.15.5:0/1007679565 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740cf010 con 0x7ff560003fb0 2023-11-24T06:17:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff572ffd640 1 -- 172.21.15.5:0/1007679565 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56000ba80 con 0x7ff5740d1670 2023-11-24T06:17:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.289+0000 7ff573fff640 1 --2- 172.21.15.5:0/1007679565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff560004d00 0x7ff560005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.290+0000 7ff5737fe640 1 --2- 172.21.15.5:0/1007679565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5740d1670 0x7ff5600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.290+0000 7ff56affd640 1 --2- 172.21.15.5:0/1007679565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff560003fb0 0x7ff5600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.290+0000 7ff573fff640 1 -- 172.21.15.5:0/1007679565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5740d1670 msgr2=0x7ff5600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.290+0000 7ff573fff640 1 --2- 172.21.15.5:0/1007679565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5740d1670 0x7ff5600039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.290+0000 7ff573fff640 1 -- 172.21.15.5:0/1007679565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff560003fb0 msgr2=0x7ff5600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.290+0000 7ff573fff640 1 --2- 172.21.15.5:0/1007679565 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff560003fb0 0x7ff5600047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.290+0000 7ff573fff640 1 -- 172.21.15.5:0/1007679565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff560003520 con 0x7ff560004d00 2023-11-24T06:17:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.290+0000 7ff5737fe640 1 --2- 172.21.15.5:0/1007679565 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff5740d1670 0x7ff5600039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.291+0000 7ff573fff640 1 --2- 172.21.15.5:0/1007679565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff560004d00 0x7ff560005180 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009ac0 tx=0x7ff5600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935395d0dbd5a414 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.291+0000 7ff572ffd640 1 -- 172.21.15.5:0/1007679565 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c02c6b0 con 0x7ff560004d00 2023-11-24T06:17:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.291+0000 7ff572ffd640 1 -- 172.21.15.5:0/1007679565 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56c02c810 con 0x7ff560004d00 2023-11-24T06:17:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.291+0000 7ff572ffd640 1 -- 172.21.15.5:0/1007679565 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c02cba0 con 0x7ff560004d00 2023-11-24T06:17:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.292+0000 7ff578f67640 1 -- 172.21.15.5:0/1007679565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff560004d00 msgr2=0x7ff560005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.292+0000 7ff578f67640 1 --2- 172.21.15.5:0/1007679565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff560004d00 0x7ff560005180 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009ac0 tx=0x7ff5600057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.292+0000 7ff573fff640 1 -- 172.21.15.5:0/1007679565 reap_dead start 2023-11-24T06:17:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.292+0000 7ff578f67640 1 -- 172.21.15.5:0/1007679565 shutdown_connections 2023-11-24T06:17:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.292+0000 7ff578f67640 1 -- 172.21.15.5:0/1007679565 >> 172.21.15.5:0/1007679565 conn(0x7ff57409c3d0 msgr2=0x7ff5740d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.292+0000 7ff578f67640 1 -- 172.21.15.5:0/1007679565 shutdown_connections 2023-11-24T06:17:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.292+0000 7ff578f67640 1 -- 172.21.15.5:0/1007679565 wait complete. 2023-11-24T06:17:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.294+0000 7ff578f67640 1 Processor -- start 2023-11-24T06:17:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.294+0000 7ff578f67640 1 -- start start 2023-11-24T06:17:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.295+0000 7ff578f67640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff560004d00 0x7ff5740cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.295+0000 7ff578f67640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff57415ac80 0x7ff57415f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.295+0000 7ff578f67640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5740cc100 0x7ff57415fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.295+0000 7ff578f67640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740b5cc0 con 0x7ff560004d00 2023-11-24T06:17:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.295+0000 7ff578f67640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740cc610 con 0x7ff5740cc100 2023-11-24T06:17:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.295+0000 7ff578f67640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740cc8f0 con 0x7ff57415ac80 2023-11-24T06:17:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.295+0000 7ff5737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff560004d00 0x7ff5740cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.295+0000 7ff5737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff560004d00 0x7ff5740cbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37168/0 (socket says 172.21.15.5:37168) 2023-11-24T06:17:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.295+0000 7ff5737fe640 1 -- 172.21.15.5:0/107884956 learned_addr learned my addr 172.21.15.5:0/107884956 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.296+0000 7ff573fff640 1 --2- 172.21.15.5:0/107884956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5740cc100 0x7ff57415fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.296+0000 7ff5737fe640 1 -- 172.21.15.5:0/107884956 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff57415ac80 msgr2=0x7ff57415f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.296+0000 7ff56affd640 1 --2- 172.21.15.5:0/107884956 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff57415ac80 0x7ff57415f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.296+0000 7ff5737fe640 1 --2- 172.21.15.5:0/107884956 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff57415ac80 0x7ff57415f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.296+0000 7ff5737fe640 1 -- 172.21.15.5:0/107884956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5740cc100 msgr2=0x7ff57415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.296+0000 7ff5737fe640 1 --2- 172.21.15.5:0/107884956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5740cc100 0x7ff57415fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.296+0000 7ff5737fe640 1 -- 172.21.15.5:0/107884956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5740ccbd0 con 0x7ff560004d00 2023-11-24T06:17:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.296+0000 7ff5737fe640 1 --2- 172.21.15.5:0/107884956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff560004d00 0x7ff5740cbbc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff564002180 tx=0x7ff56400b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.296+0000 7ff570ff9640 1 -- 172.21.15.5:0/107884956 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564004d60 con 0x7ff560004d00 2023-11-24T06:17:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.296+0000 7ff570ff9640 1 -- 172.21.15.5:0/107884956 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5640070d0 con 0x7ff560004d00 2023-11-24T06:17:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.296+0000 7ff570ff9640 1 -- 172.21.15.5:0/107884956 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff564007460 con 0x7ff560004d00 2023-11-24T06:17:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.296+0000 7ff578f67640 1 -- 172.21.15.5:0/107884956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5740b5950 con 0x7ff560004d00 2023-11-24T06:17:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.296+0000 7ff578f67640 1 -- 172.21.15.5:0/107884956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff57415d9b0 con 0x7ff560004d00 2023-11-24T06:17:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.298+0000 7ff578f67640 1 -- 172.21.15.5:0/107884956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5740b1c00 con 0x7ff560004d00 2023-11-24T06:17:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.298+0000 7ff570ff9640 1 -- 172.21.15.5:0/107884956 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ff5640075c0 con 0x7ff560004d00 2023-11-24T06:17:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.300+0000 7ff570ff9640 1 --2- 172.21.15.5:0/107884956 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff55406af00 0x7ff55406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.300+0000 7ff570ff9640 1 -- 172.21.15.5:0/107884956 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff564089920 con 0x7ff560004d00 2023-11-24T06:17:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.300+0000 7ff56affd640 1 --2- 172.21.15.5:0/107884956 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff55406af00 0x7ff55406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.301+0000 7ff56affd640 1 --2- 172.21.15.5:0/107884956 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff55406af00 0x7ff55406d3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff574160b60 tx=0x7ff55c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.304+0000 7ff570ff9640 1 -- 172.21.15.5:0/107884956 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff564054860 con 0x7ff560004d00 2023-11-24T06:17:36.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.511+0000 7ff578f67640 1 -- 172.21.15.5:0/107884956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff57404e840 con 0x7ff560004d00 2023-11-24T06:17:36.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.512+0000 7ff570ff9640 1 -- 172.21.15.5:0/107884956 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff564019080 con 0x7ff560004d00 2023-11-24T06:17:36.515 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2023-11-24T06:17:36.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.514+0000 7ff578f67640 1 -- 172.21.15.5:0/107884956 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff55406af00 msgr2=0x7ff55406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:36.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.514+0000 7ff578f67640 1 --2- 172.21.15.5:0/107884956 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff55406af00 0x7ff55406d3c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff574160b60 tx=0x7ff55c008040 comp rx=0 tx=0).stop 2023-11-24T06:17:36.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.514+0000 7ff578f67640 1 -- 172.21.15.5:0/107884956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff560004d00 msgr2=0x7ff5740cbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:36.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.514+0000 7ff578f67640 1 --2- 172.21.15.5:0/107884956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff560004d00 0x7ff5740cbbc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff564002180 tx=0x7ff56400b560 comp rx=0 tx=0).stop 2023-11-24T06:17:36.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.515+0000 7ff578f67640 1 -- 172.21.15.5:0/107884956 shutdown_connections 2023-11-24T06:17:36.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.515+0000 7ff578f67640 1 --2- 172.21.15.5:0/107884956 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff55406af00 0x7ff55406d3c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.515+0000 7ff578f67640 1 --2- 172.21.15.5:0/107884956 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff57415ac80 0x7ff57415f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.515+0000 7ff578f67640 1 --2- 172.21.15.5:0/107884956 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff5740cc100 0x7ff57415fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.515+0000 7ff578f67640 1 --2- 172.21.15.5:0/107884956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff560004d00 0x7ff5740cbbc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:36.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.515+0000 7ff578f67640 1 -- 172.21.15.5:0/107884956 >> 172.21.15.5:0/107884956 conn(0x7ff57409c3d0 msgr2=0x7ff5740b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:36.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.515+0000 7ff578f67640 1 -- 172.21.15.5:0/107884956 shutdown_connections 2023-11-24T06:17:36.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:36.515+0000 7ff578f67640 1 -- 172.21.15.5:0/107884956 wait complete. 2023-11-24T06:17:36.535 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi005.front.sepia.ceph.com... 2023-11-24T06:17:36.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:36.535 DEBUG:teuthology.orchestra.run.smithi005:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-24T06:17:36.627 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[101461]: fuse finished with error 0 and tester_r 0 2023-11-24T06:17:36.633 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T06:17:36.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:36.633 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T06:17:36.755 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:17:36.755 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:17:36.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:17:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.051+0000 7f527c778640 1 Processor -- start 2023-11-24T06:17:37.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.051+0000 7f527c778640 1 -- start start 2023-11-24T06:17:37.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.053+0000 7f527c778640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5274070780 0x7f527406d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:37.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.053+0000 7f527c778640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5274070cd0 0x7f527406da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:37.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.053+0000 7f527c778640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5274071cd0 con 0x7f5274160970 2023-11-24T06:17:37.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.053+0000 7f527c778640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52740b2120 con 0x7f5274070780 2023-11-24T06:17:37.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.053+0000 7f527c778640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52740b2400 con 0x7f5274070cd0 2023-11-24T06:17:37.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.054+0000 7f527ad15640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5274070cd0 0x7f527406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:37.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.054+0000 7f527ad15640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5274070cd0 0x7f527406da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:60606/0 (socket says 172.21.15.5:60606) 2023-11-24T06:17:37.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.054+0000 7f527ad15640 1 -- 172.21.15.5:0/2088780128 learned_addr learned my addr 172.21.15.5:0/2088780128 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:37.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.054+0000 7f5279d13640 1 --2- 172.21.15.5:0/2088780128 >> v2:172.21.15.6:3300/0 conn(0x7f5274070780 0x7f527406d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:37.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.055+0000 7f527ad15640 1 -- 172.21.15.5:0/2088780128 >> v2:172.21.15.6:3300/0 conn(0x7f5274070780 msgr2=0x7f527406d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:37.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.055+0000 7f527ad15640 1 --2- 172.21.15.5:0/2088780128 >> v2:172.21.15.6:3300/0 conn(0x7f5274070780 0x7f527406d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.055+0000 7f527ad15640 1 -- 172.21.15.5:0/2088780128 >> v1:172.21.15.6:6789/0 conn(0x7f5274160970 legacy=0x7f52740b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:37.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.055+0000 7f527ad15640 1 -- 172.21.15.5:0/2088780128 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52740b26e0 con 0x7f5274070cd0 2023-11-24T06:17:37.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.057+0000 7f527ad15640 1 --2- 172.21.15.5:0/2088780128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5274070cd0 0x7f527406da60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f526c009e50 tx=0x7f526c02bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c8fa07d60a65461 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:37.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.057+0000 7f5279512640 1 -- 172.21.15.5:0/2088780128 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c02ca80 con 0x7f5274070cd0 2023-11-24T06:17:37.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.057+0000 7f5279512640 1 -- 172.21.15.5:0/2088780128 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526c02cbe0 con 0x7f5274070cd0 2023-11-24T06:17:37.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.057+0000 7f5279512640 1 -- 172.21.15.5:0/2088780128 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c0343f0 con 0x7f5274070cd0 2023-11-24T06:17:37.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.058+0000 7f527c778640 1 -- 172.21.15.5:0/2088780128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5274070cd0 msgr2=0x7f527406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:37.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.058+0000 7f527c778640 1 --2- 172.21.15.5:0/2088780128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5274070cd0 0x7f527406da60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f526c009e50 tx=0x7f526c02bee0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.058+0000 7f527c778640 1 -- 172.21.15.5:0/2088780128 shutdown_connections 2023-11-24T06:17:37.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.058+0000 7f527c778640 1 --2- 172.21.15.5:0/2088780128 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5274070cd0 0x7f527406da60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.058+0000 7f527c778640 1 --2- 172.21.15.5:0/2088780128 >> v2:172.21.15.6:3300/0 conn(0x7f5274070780 0x7f527406d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.058+0000 7f527c778640 1 -- 172.21.15.5:0/2088780128 >> 172.21.15.5:0/2088780128 conn(0x7f52740b0d80 msgr2=0x7f52740b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:37.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.058+0000 7f527c778640 1 -- 172.21.15.5:0/2088780128 shutdown_connections 2023-11-24T06:17:37.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.058+0000 7f527c778640 1 -- 172.21.15.5:0/2088780128 wait complete. 2023-11-24T06:17:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.061+0000 7f527c778640 1 Processor -- start 2023-11-24T06:17:37.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.061+0000 7f527c778640 1 -- start start 2023-11-24T06:17:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.062+0000 7f527c778640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5274070780 0x7f5274150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.062+0000 7f527c778640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5274070cd0 0x7f52741596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.062+0000 7f527c778640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5274160970 0x7f5274099e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.062+0000 7f527c778640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527409cd00 con 0x7f5274070780 2023-11-24T06:17:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.062+0000 7f527c778640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5274150ba0 con 0x7f5274160970 2023-11-24T06:17:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.062+0000 7f527c778640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527409be50 con 0x7f5274070cd0 2023-11-24T06:17:37.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.062+0000 7f527a514640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5274070780 0x7f5274150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.062+0000 7f527a514640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5274070780 0x7f5274150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37182/0 (socket says 172.21.15.5:37182) 2023-11-24T06:17:37.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.062+0000 7f527a514640 1 -- 172.21.15.5:0/623763986 learned_addr learned my addr 172.21.15.5:0/623763986 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.063+0000 7f527ad15640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5274160970 0x7f5274099e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.063+0000 7f5279d13640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5274070cd0 0x7f52741596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.063+0000 7f527a514640 1 -- 172.21.15.5:0/623763986 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5274070cd0 msgr2=0x7f52741596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.063+0000 7f527a514640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5274070cd0 0x7f52741596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.063+0000 7f527a514640 1 -- 172.21.15.5:0/623763986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5274160970 msgr2=0x7f5274099e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.063+0000 7f527a514640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5274160970 0x7f5274099e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.063+0000 7f527a514640 1 -- 172.21.15.5:0/623763986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c009b30 con 0x7f5274070780 2023-11-24T06:17:37.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.063+0000 7f527ad15640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5274160970 0x7f5274099e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.063+0000 7f527a514640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5274070780 0x7f5274150660 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f527000c400 tx=0x7f527000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.064+0000 7f526affd640 1 -- 172.21.15.5:0/623763986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52700052d0 con 0x7f5274070780 2023-11-24T06:17:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.064+0000 7f526affd640 1 -- 172.21.15.5:0/623763986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5270005450 con 0x7f5274070780 2023-11-24T06:17:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.064+0000 7f527c778640 1 -- 172.21.15.5:0/623763986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527409d120 con 0x7f5274070780 2023-11-24T06:17:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.064+0000 7f526affd640 1 -- 172.21.15.5:0/623763986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270007260 con 0x7f5274070780 2023-11-24T06:17:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.064+0000 7f527c778640 1 -- 172.21.15.5:0/623763986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527409c360 con 0x7f5274070780 2023-11-24T06:17:37.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.065+0000 7f527c778640 1 -- 172.21.15.5:0/623763986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527409ce40 con 0x7f5274070780 2023-11-24T06:17:37.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.066+0000 7f526affd640 1 -- 172.21.15.5:0/623763986 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f52700073c0 con 0x7f5274070780 2023-11-24T06:17:37.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.068+0000 7f526affd640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f525806aed0 0x7f525806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:37.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.068+0000 7f526affd640 1 -- 172.21.15.5:0/623763986 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5270088af0 con 0x7f5274070780 2023-11-24T06:17:37.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.068+0000 7f5279d13640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f525806aed0 0x7f525806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:37.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.069+0000 7f5279d13640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f525806aed0 0x7f525806d390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f527409d340 tx=0x7f5264008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:37.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.074+0000 7f526affd640 1 -- 172.21.15.5:0/623763986 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f527409ce40 con 0x7f5274070780 2023-11-24T06:17:37.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.275+0000 7f527c778640 1 -- 172.21.15.5:0/623763986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f527409ce40 con 0x7f5274070780 2023-11-24T06:17:37.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.277+0000 7f526affd640 1 -- 172.21.15.5:0/623763986 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16425 (secure 0 0 0) 0x7f527409ce40 con 0x7f5274070780 2023-11-24T06:17:37.281 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:37.281 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:17:18.380558+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:17:14.868917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:17:15.895189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:17:37.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.280+0000 7f527c778640 1 -- 172.21.15.5:0/623763986 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f525806aed0 msgr2=0x7f525806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:37.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.280+0000 7f527c778640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f525806aed0 0x7f525806d390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f527409d340 tx=0x7f5264008040 comp rx=0 tx=0).stop 2023-11-24T06:17:37.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.280+0000 7f527c778640 1 -- 172.21.15.5:0/623763986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5274070780 msgr2=0x7f5274150660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:37.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.280+0000 7f527c778640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5274070780 0x7f5274150660 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f527000c400 tx=0x7f527000c8c0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.280+0000 7f527c778640 1 -- 172.21.15.5:0/623763986 shutdown_connections 2023-11-24T06:17:37.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.280+0000 7f527c778640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f525806aed0 0x7f525806d390 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.280+0000 7f527c778640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5274070cd0 0x7f52741596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.280+0000 7f527c778640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5274160970 0x7f5274099e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.280+0000 7f527c778640 1 --2- 172.21.15.5:0/623763986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5274070780 0x7f5274150660 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.280+0000 7f527c778640 1 -- 172.21.15.5:0/623763986 >> 172.21.15.5:0/623763986 conn(0x7f52740b0d80 msgr2=0x7f527415a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:37.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.281+0000 7f527c778640 1 -- 172.21.15.5:0/623763986 shutdown_connections 2023-11-24T06:17:37.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.281+0000 7f527c778640 1 -- 172.21.15.5:0/623763986 wait complete. 2023-11-24T06:17:37.303 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:17:14.868917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:17:15.895189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:17:37.303 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:17:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.531+0000 7f28048a5640 1 Processor -- start 2023-11-24T06:17:37.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.531+0000 7f28048a5640 1 -- start start 2023-11-24T06:17:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.532+0000 7f28048a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28000d1fd0 0x7f28000ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.532+0000 7f28048a5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f28000b33d0 0x7f28000b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.532+0000 7f28048a5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28000b2980 con 0x7f28000cfa30 2023-11-24T06:17:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.532+0000 7f28048a5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28000cf220 con 0x7f28000b33d0 2023-11-24T06:17:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.532+0000 7f28048a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28000cf500 con 0x7f28000d1fd0 2023-11-24T06:17:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.533+0000 7f27fed9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f28000cfa30 0x7f28000cfe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35666/0 (socket says 172.21.15.5:35666) 2023-11-24T06:17:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.533+0000 7f27fed9d640 1 -- 172.21.15.5:0/1311788767 learned_addr learned my addr 172.21.15.5:0/1311788767 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.533+0000 7f27ff59e640 1 --2- 172.21.15.5:0/1311788767 >> v2:172.21.15.5:3300/0 conn(0x7f28000b33d0 0x7f28000b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.533+0000 7f27fe59c640 1 --2- 172.21.15.5:0/1311788767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28000d1fd0 0x7f28000ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.533+0000 7f27fdd9b640 1 -- 172.21.15.5:0/1311788767 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49625326 0 0) 0x7f28000b2980 con 0x7f28000cfa30 2023-11-24T06:17:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.533+0000 7f27fdd9b640 1 -- 172.21.15.5:0/1311788767 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27e8003520 con 0x7f28000cfa30 2023-11-24T06:17:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.533+0000 7f27ff59e640 1 -- 172.21.15.5:0/1311788767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28000d1fd0 msgr2=0x7f28000ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.533+0000 7f27ff59e640 1 --2- 172.21.15.5:0/1311788767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28000d1fd0 0x7f28000ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.533+0000 7f27ff59e640 1 -- 172.21.15.5:0/1311788767 >> v1:172.21.15.5:6789/0 conn(0x7f28000cfa30 legacy=0x7f28000cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.533+0000 7f27ff59e640 1 -- 172.21.15.5:0/1311788767 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28000b2bd0 con 0x7f28000b33d0 2023-11-24T06:17:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.535+0000 7f27ff59e640 1 --2- 172.21.15.5:0/1311788767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28000b33d0 0x7f28000b3860 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f27f80097a0 tx=0x7f27f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5dee51841b5947 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.535+0000 7f27fdd9b640 1 -- 172.21.15.5:0/1311788767 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8031830 con 0x7f28000b33d0 2023-11-24T06:17:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.535+0000 7f27fdd9b640 1 -- 172.21.15.5:0/1311788767 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f8031990 con 0x7f28000b33d0 2023-11-24T06:17:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.535+0000 7f27fdd9b640 1 -- 172.21.15.5:0/1311788767 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8031d20 con 0x7f28000b33d0 2023-11-24T06:17:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.536+0000 7f28048a5640 1 -- 172.21.15.5:0/1311788767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28000b33d0 msgr2=0x7f28000b3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.536+0000 7f28048a5640 1 --2- 172.21.15.5:0/1311788767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28000b33d0 0x7f28000b3860 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f27f80097a0 tx=0x7f27f802b8d0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.536+0000 7f28048a5640 1 -- 172.21.15.5:0/1311788767 shutdown_connections 2023-11-24T06:17:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.536+0000 7f28048a5640 1 --2- 172.21.15.5:0/1311788767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28000b33d0 0x7f28000b3860 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.536+0000 7f28048a5640 1 --2- 172.21.15.5:0/1311788767 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28000d1fd0 0x7f28000ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.536+0000 7f28048a5640 1 -- 172.21.15.5:0/1311788767 >> 172.21.15.5:0/1311788767 conn(0x7f28000d3020 msgr2=0x7f28000d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.536+0000 7f28048a5640 1 -- 172.21.15.5:0/1311788767 shutdown_connections 2023-11-24T06:17:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.536+0000 7f28048a5640 1 -- 172.21.15.5:0/1311788767 wait complete. 2023-11-24T06:17:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.538+0000 7f28048a5640 1 Processor -- start 2023-11-24T06:17:37.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.538+0000 7f28048a5640 1 -- start start 2023-11-24T06:17:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.539+0000 7f28048a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28000b33d0 0x7f2800152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.539+0000 7f28048a5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28000cfa30 0x7f28000c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.539+0000 7f28048a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28000d1fd0 0x7f28000d1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.539+0000 7f28048a5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28000c8f60 con 0x7f28000d1fd0 2023-11-24T06:17:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.539+0000 7f28048a5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28000c8c80 con 0x7f28000cfa30 2023-11-24T06:17:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.539+0000 7f28048a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28000b5ce0 con 0x7f28000b33d0 2023-11-24T06:17:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.539+0000 7f27ff59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28000d1fd0 0x7f28000d1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:37.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.539+0000 7f27ff59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28000d1fd0 0x7f28000d1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37198/0 (socket says 172.21.15.5:37198) 2023-11-24T06:17:37.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.539+0000 7f27ff59e640 1 -- 172.21.15.5:0/3300535584 learned_addr learned my addr 172.21.15.5:0/3300535584 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.540+0000 7f27fed9d640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28000b33d0 0x7f2800152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.540+0000 7f27fe59c640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28000cfa30 0x7f28000c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.540+0000 7f27ff59e640 1 -- 172.21.15.5:0/3300535584 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28000b33d0 msgr2=0x7f2800152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.540+0000 7f27ff59e640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28000b33d0 0x7f2800152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.540+0000 7f27ff59e640 1 -- 172.21.15.5:0/3300535584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28000cfa30 msgr2=0x7f28000c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.540+0000 7f27ff59e640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28000cfa30 0x7f28000c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.540+0000 7f27ff59e640 1 -- 172.21.15.5:0/3300535584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f8009400 con 0x7f28000d1fd0 2023-11-24T06:17:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.540+0000 7f27fe59c640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28000cfa30 0x7f28000c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:37.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.540+0000 7f27ff59e640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28000d1fd0 0x7f28000d1020 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f27f80096f0 tx=0x7f27f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.541+0000 7f27e77fe640 1 -- 172.21.15.5:0/3300535584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f803a8f0 con 0x7f28000d1fd0 2023-11-24T06:17:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.541+0000 7f27e77fe640 1 -- 172.21.15.5:0/3300535584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f803aa50 con 0x7f28000d1fd0 2023-11-24T06:17:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.541+0000 7f27e77fe640 1 -- 172.21.15.5:0/3300535584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f803ae00 con 0x7f28000d1fd0 2023-11-24T06:17:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.541+0000 7f28048a5640 1 -- 172.21.15.5:0/3300535584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28000b5fc0 con 0x7f28000d1fd0 2023-11-24T06:17:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.541+0000 7f28048a5640 1 -- 172.21.15.5:0/3300535584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28000c8970 con 0x7f28000d1fd0 2023-11-24T06:17:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.542+0000 7f28048a5640 1 -- 172.21.15.5:0/3300535584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28000b64e0 con 0x7f28000d1fd0 2023-11-24T06:17:37.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.543+0000 7f27e77fe640 1 -- 172.21.15.5:0/3300535584 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f27f804d440 con 0x7f28000d1fd0 2023-11-24T06:17:37.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.544+0000 7f27e77fe640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f27ec06aed0 0x7f27ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:37.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.544+0000 7f27e77fe640 1 -- 172.21.15.5:0/3300535584 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27f80ab980 con 0x7f28000d1fd0 2023-11-24T06:17:37.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.544+0000 7f27fed9d640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f27ec06aed0 0x7f27ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:37.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.545+0000 7f27fed9d640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f27ec06aed0 0x7f27ec06d390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f27f0001560 tx=0x7f27f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:37.550 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:17:37.554 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:17:37.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.549+0000 7f27e77fe640 1 -- 172.21.15.5:0/3300535584 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28000b64e0 con 0x7f28000d1fd0 2023-11-24T06:17:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.745+0000 7f28048a5640 1 -- 172.21.15.5:0/3300535584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28000b64e0 con 0x7f28000d1fd0 2023-11-24T06:17:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.746+0000 7f27e77fe640 1 -- 172.21.15.5:0/3300535584 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28000b64e0 con 0x7f28000d1fd0 2023-11-24T06:17:37.749 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:17:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.749+0000 7f28048a5640 1 -- 172.21.15.5:0/3300535584 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f27ec06aed0 msgr2=0x7f27ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.749+0000 7f28048a5640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f27ec06aed0 0x7f27ec06d390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f27f0001560 tx=0x7f27f0009040 comp rx=0 tx=0).stop 2023-11-24T06:17:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.749+0000 7f28048a5640 1 -- 172.21.15.5:0/3300535584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28000d1fd0 msgr2=0x7f28000d1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.749+0000 7f28048a5640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28000d1fd0 0x7f28000d1020 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f27f80096f0 tx=0x7f27f80046c0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.749+0000 7f28048a5640 1 -- 172.21.15.5:0/3300535584 shutdown_connections 2023-11-24T06:17:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.749+0000 7f28048a5640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f27ec06aed0 0x7f27ec06d390 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.749+0000 7f28048a5640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28000d1fd0 0x7f28000d1020 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.749+0000 7f28048a5640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f28000cfa30 0x7f28000c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.749+0000 7f28048a5640 1 --2- 172.21.15.5:0/3300535584 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f28000b33d0 0x7f2800152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.749+0000 7f28048a5640 1 -- 172.21.15.5:0/3300535584 >> 172.21.15.5:0/3300535584 conn(0x7f28000d3020 msgr2=0x7f28000d3400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.749+0000 7f28048a5640 1 -- 172.21.15.5:0/3300535584 shutdown_connections 2023-11-24T06:17:37.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.749+0000 7f28048a5640 1 -- 172.21.15.5:0/3300535584 wait complete. 2023-11-24T06:17:37.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:17:38.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.997+0000 7fdf8ab2e640 1 Processor -- start 2023-11-24T06:17:38.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.997+0000 7fdf8ab2e640 1 -- start start 2023-11-24T06:17:38.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.999+0000 7fdf8ab2e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf84144770 0x7fdf8415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.999+0000 7fdf8ab2e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf8415f930 con 0x7fdf8415f0a0 2023-11-24T06:17:38.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.999+0000 7fdf8ab2e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf840b3b10 con 0x7fdf840b35a0 2023-11-24T06:17:38.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.999+0000 7fdf8ab2e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf840b43b0 con 0x7fdf84144770 2023-11-24T06:17:38.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.999+0000 7fdf8a32d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fdf8415f0a0 0x7fdf840ce4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35674/0 (socket says 172.21.15.5:35674) 2023-11-24T06:17:38.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:37.999+0000 7fdf8a32d640 1 -- 172.21.15.5:0/2511351618 learned_addr learned my addr 172.21.15.5:0/2511351618 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:38.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.000+0000 7fdf89b2c640 1 --2- 172.21.15.5:0/2511351618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf84144770 0x7fdf8415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.000+0000 7fdf88b2a640 1 -- 172.21.15.5:0/2511351618 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437669079 0 0) 0x7fdf8415f930 con 0x7fdf8415f0a0 2023-11-24T06:17:38.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.000+0000 7fdf88b2a640 1 -- 172.21.15.5:0/2511351618 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf6c003520 con 0x7fdf8415f0a0 2023-11-24T06:17:38.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.000+0000 7fdf88b2a640 1 -- 172.21.15.5:0/2511351618 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4127928425 0 0) 0x7fdf6c003520 con 0x7fdf8415f0a0 2023-11-24T06:17:38.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.000+0000 7fdf88b2a640 1 -- 172.21.15.5:0/2511351618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf84144770 msgr2=0x7fdf8415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:38.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.000+0000 7fdf88b2a640 1 --2- 172.21.15.5:0/2511351618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf84144770 0x7fdf8415eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.000+0000 7fdf88b2a640 1 -- 172.21.15.5:0/2511351618 >> v1:172.21.15.6:6789/0 conn(0x7fdf840b35a0 legacy=0x7fdf8406c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.000+0000 7fdf88b2a640 1 -- 172.21.15.5:0/2511351618 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf840b3df0 con 0x7fdf8415f0a0 2023-11-24T06:17:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.000+0000 7fdf88b2a640 1 -- 172.21.15.5:0/2511351618 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fdf7c001670 con 0x7fdf8415f0a0 2023-11-24T06:17:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.000+0000 7fdf88b2a640 1 -- 172.21.15.5:0/2511351618 >> v1:172.21.15.5:6789/0 conn(0x7fdf8415f0a0 legacy=0x7fdf840ce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.000+0000 7fdf88b2a640 1 --2- 172.21.15.5:0/2511351618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf6c003a10 0x7fdf6c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.000+0000 7fdf88b2a640 1 --2- 172.21.15.5:0/2511351618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf84144770 0x7fdf6c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.001+0000 7fdf88b2a640 1 --2- 172.21.15.5:0/2511351618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c004d00 0x7fdf6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.001+0000 7fdf88b2a640 1 -- 172.21.15.5:0/2511351618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf840b43b0 con 0x7fdf6c004d00 2023-11-24T06:17:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.001+0000 7fdf88b2a640 1 -- 172.21.15.5:0/2511351618 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c00ba90 con 0x7fdf6c003a10 2023-11-24T06:17:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.001+0000 7fdf88b2a640 1 -- 172.21.15.5:0/2511351618 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c00bc50 con 0x7fdf84144770 2023-11-24T06:17:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.001+0000 7fdf8a32d640 1 --2- 172.21.15.5:0/2511351618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c004d00 0x7fdf6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.001+0000 7fdf8932b640 1 --2- 172.21.15.5:0/2511351618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf84144770 0x7fdf6c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.001+0000 7fdf89b2c640 1 --2- 172.21.15.5:0/2511351618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf6c003a10 0x7fdf6c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.001+0000 7fdf8a32d640 1 -- 172.21.15.5:0/2511351618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf84144770 msgr2=0x7fdf6c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:38.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.001+0000 7fdf8a32d640 1 --2- 172.21.15.5:0/2511351618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf84144770 0x7fdf6c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.001+0000 7fdf8a32d640 1 -- 172.21.15.5:0/2511351618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf6c003a10 msgr2=0x7fdf6c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:38.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.001+0000 7fdf8a32d640 1 --2- 172.21.15.5:0/2511351618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf6c003a10 0x7fdf6c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.001+0000 7fdf8a32d640 1 -- 172.21.15.5:0/2511351618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6c003520 con 0x7fdf6c004d00 2023-11-24T06:17:38.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.002+0000 7fdf8a32d640 1 --2- 172.21.15.5:0/2511351618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c004d00 0x7fdf6c005180 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00c210 tx=0x7fdf6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2007133ae10d729 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:38.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.003+0000 7fdf88b2a640 1 -- 172.21.15.5:0/2511351618 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c005ad0 con 0x7fdf6c004d00 2023-11-24T06:17:38.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.003+0000 7fdf88b2a640 1 -- 172.21.15.5:0/2511351618 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c005c30 con 0x7fdf6c004d00 2023-11-24T06:17:38.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.003+0000 7fdf88b2a640 1 -- 172.21.15.5:0/2511351618 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c02c890 con 0x7fdf6c004d00 2023-11-24T06:17:38.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.003+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2511351618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c004d00 msgr2=0x7fdf6c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.003+0000 7fdf8ab2e640 1 --2- 172.21.15.5:0/2511351618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c004d00 0x7fdf6c005180 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00c210 tx=0x7fdf6c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.003+0000 7fdf8a32d640 1 -- 172.21.15.5:0/2511351618 reap_dead start 2023-11-24T06:17:38.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.004+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2511351618 shutdown_connections 2023-11-24T06:17:38.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.004+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2511351618 >> 172.21.15.5:0/2511351618 conn(0x7fdf840cec10 msgr2=0x7fdf840cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:38.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.004+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2511351618 shutdown_connections 2023-11-24T06:17:38.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.004+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2511351618 wait complete. 2023-11-24T06:17:38.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.005+0000 7fdf8ab2e640 1 Processor -- start 2023-11-24T06:17:38.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.006+0000 7fdf8ab2e640 1 -- start start 2023-11-24T06:17:38.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.007+0000 7fdf8ab2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c004d00 0x7fdf840b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.007+0000 7fdf8ab2e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf840cb9c0 0x7fdf840c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.007+0000 7fdf8ab2e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf840cbe40 0x7fdf840c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.007+0000 7fdf8ab2e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf84140d60 con 0x7fdf6c004d00 2023-11-24T06:17:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.007+0000 7fdf8ab2e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf84140770 con 0x7fdf840cb9c0 2023-11-24T06:17:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.007+0000 7fdf8ab2e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf84152f90 con 0x7fdf840cbe40 2023-11-24T06:17:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.007+0000 7fdf89b2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c004d00 0x7fdf840b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.007+0000 7fdf89b2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c004d00 0x7fdf840b0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37222/0 (socket says 172.21.15.5:37222) 2023-11-24T06:17:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.007+0000 7fdf89b2c640 1 -- 172.21.15.5:0/2474436893 learned_addr learned my addr 172.21.15.5:0/2474436893 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.007+0000 7fdf8932b640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf840cb9c0 0x7fdf840c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.008+0000 7fdf8a32d640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf840cbe40 0x7fdf840c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.008+0000 7fdf89b2c640 1 -- 172.21.15.5:0/2474436893 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf840cbe40 msgr2=0x7fdf840c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.008+0000 7fdf89b2c640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf840cbe40 0x7fdf840c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.008+0000 7fdf89b2c640 1 -- 172.21.15.5:0/2474436893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf840cb9c0 msgr2=0x7fdf840c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.008+0000 7fdf89b2c640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf840cb9c0 0x7fdf840c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.008+0000 7fdf89b2c640 1 -- 172.21.15.5:0/2474436893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf840c8e30 con 0x7fdf6c004d00 2023-11-24T06:17:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.008+0000 7fdf8a32d640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf840cbe40 0x7fdf840c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.008+0000 7fdf89b2c640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c004d00 0x7fdf840b0ed0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdf74007f30 tx=0x7fdf74011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.008+0000 7fdf827fc640 1 -- 172.21.15.5:0/2474436893 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7400e030 con 0x7fdf6c004d00 2023-11-24T06:17:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.008+0000 7fdf827fc640 1 -- 172.21.15.5:0/2474436893 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf74016470 con 0x7fdf6c004d00 2023-11-24T06:17:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.008+0000 7fdf827fc640 1 -- 172.21.15.5:0/2474436893 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf74016820 con 0x7fdf6c004d00 2023-11-24T06:17:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.008+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2474436893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf840c9110 con 0x7fdf6c004d00 2023-11-24T06:17:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.009+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2474436893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf840c8a50 con 0x7fdf6c004d00 2023-11-24T06:17:38.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.010+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2474436893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf840c8be0 con 0x7fdf6c004d00 2023-11-24T06:17:38.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.011+0000 7fdf827fc640 1 -- 172.21.15.5:0/2474436893 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fdf74016980 con 0x7fdf6c004d00 2023-11-24T06:17:38.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.012+0000 7fdf827fc640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf7006aed0 0x7fdf7006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.013+0000 7fdf827fc640 1 -- 172.21.15.5:0/2474436893 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf7400f070 con 0x7fdf6c004d00 2023-11-24T06:17:38.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.013+0000 7fdf8932b640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf7006aed0 0x7fdf7006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.014+0000 7fdf8932b640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf7006aed0 0x7fdf7006d390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdf841532b0 tx=0x7fdf78009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:38.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.017+0000 7fdf827fc640 1 -- 172.21.15.5:0/2474436893 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdf740556c0 con 0x7fdf6c004d00 2023-11-24T06:17:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.214+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2474436893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf840c9300 con 0x7fdf6c004d00 2023-11-24T06:17:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.215+0000 7fdf827fc640 1 -- 172.21.15.5:0/2474436893 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdf74058ce0 con 0x7fdf6c004d00 2023-11-24T06:17:38.218 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:17:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.217+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2474436893 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf7006aed0 msgr2=0x7fdf7006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.217+0000 7fdf8ab2e640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf7006aed0 0x7fdf7006d390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdf841532b0 tx=0x7fdf78009040 comp rx=0 tx=0).stop 2023-11-24T06:17:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.217+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2474436893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c004d00 msgr2=0x7fdf840b0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.217+0000 7fdf8ab2e640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c004d00 0x7fdf840b0ed0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdf74007f30 tx=0x7fdf74011660 comp rx=0 tx=0).stop 2023-11-24T06:17:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.218+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2474436893 shutdown_connections 2023-11-24T06:17:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.218+0000 7fdf8ab2e640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fdf7006aed0 0x7fdf7006d390 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.218+0000 7fdf8ab2e640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fdf840cbe40 0x7fdf840c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.218+0000 7fdf8ab2e640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fdf840cb9c0 0x7fdf840c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.218+0000 7fdf8ab2e640 1 --2- 172.21.15.5:0/2474436893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf6c004d00 0x7fdf840b0ed0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.218+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2474436893 >> 172.21.15.5:0/2474436893 conn(0x7fdf840cec10 msgr2=0x7fdf840cd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.218+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2474436893 shutdown_connections 2023-11-24T06:17:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.218+0000 7fdf8ab2e640 1 -- 172.21.15.5:0/2474436893 wait complete. 2023-11-24T06:17:38.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:17:38.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.454+0000 7f01f152e640 1 Processor -- start 2023-11-24T06:17:38.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.454+0000 7f01f152e640 1 -- start start 2023-11-24T06:17:38.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.455+0000 7f01f152e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f01ec15d3b0 0x7f01ec0ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.455+0000 7f01f152e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f01ec0d0fa0 0x7f01ec0d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.455+0000 7f01f152e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01ec0c6bf0 con 0x7f01ec0d1bc0 2023-11-24T06:17:38.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.456+0000 7f01f152e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ec0d1430 con 0x7f01ec15d3b0 2023-11-24T06:17:38.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.456+0000 7f01f152e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ec0d1fa0 con 0x7f01ec0d0fa0 2023-11-24T06:17:38.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.456+0000 7f01ebfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f01ec15d3b0 0x7f01ec0ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.456+0000 7f01ebfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f01ec15d3b0 0x7f01ec0ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37234/0 (socket says 172.21.15.5:37234) 2023-11-24T06:17:38.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.456+0000 7f01ebfff640 1 -- 172.21.15.5:0/4111907481 learned_addr learned my addr 172.21.15.5:0/4111907481 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:38.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.456+0000 7f01eb7fe640 1 --2- 172.21.15.5:0/4111907481 >> v2:172.21.15.6:3300/0 conn(0x7f01ec0d0fa0 0x7f01ec0d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.456+0000 7f01ebfff640 1 -- 172.21.15.5:0/4111907481 >> v2:172.21.15.6:3300/0 conn(0x7f01ec0d0fa0 msgr2=0x7f01ec0d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.456+0000 7f01ebfff640 1 --2- 172.21.15.5:0/4111907481 >> v2:172.21.15.6:3300/0 conn(0x7f01ec0d0fa0 0x7f01ec0d1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.456+0000 7f01ebfff640 1 -- 172.21.15.5:0/4111907481 >> v1:172.21.15.6:6789/0 conn(0x7f01ec0d1bc0 legacy=0x7f01ec0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.456+0000 7f01ebfff640 1 -- 172.21.15.5:0/4111907481 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec0cedc0 con 0x7f01ec15d3b0 2023-11-24T06:17:38.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.458+0000 7f01ebfff640 1 --2- 172.21.15.5:0/4111907481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ec15d3b0 0x7f01ec0ce6c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f01ec08c4a0 tx=0x7f01dc02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542e1405e99238ae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:38.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.458+0000 7f01eaffd640 1 -- 172.21.15.5:0/4111907481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc02c9f0 con 0x7f01ec15d3b0 2023-11-24T06:17:38.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.458+0000 7f01eaffd640 1 -- 172.21.15.5:0/4111907481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc02cb50 con 0x7f01ec15d3b0 2023-11-24T06:17:38.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.458+0000 7f01eaffd640 1 -- 172.21.15.5:0/4111907481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc0343f0 con 0x7f01ec15d3b0 2023-11-24T06:17:38.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.459+0000 7f01f152e640 1 -- 172.21.15.5:0/4111907481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ec15d3b0 msgr2=0x7f01ec0ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.459+0000 7f01f152e640 1 --2- 172.21.15.5:0/4111907481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ec15d3b0 0x7f01ec0ce6c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f01ec08c4a0 tx=0x7f01dc02be50 comp rx=0 tx=0).stop 2023-11-24T06:17:38.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.459+0000 7f01f152e640 1 -- 172.21.15.5:0/4111907481 shutdown_connections 2023-11-24T06:17:38.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.459+0000 7f01f152e640 1 --2- 172.21.15.5:0/4111907481 >> v2:172.21.15.6:3300/0 conn(0x7f01ec0d0fa0 0x7f01ec0d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.459+0000 7f01f152e640 1 --2- 172.21.15.5:0/4111907481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ec15d3b0 0x7f01ec0ce6c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.459+0000 7f01f152e640 1 -- 172.21.15.5:0/4111907481 >> 172.21.15.5:0/4111907481 conn(0x7f01ec0d30a0 msgr2=0x7f01ec0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:38.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.459+0000 7f01f152e640 1 -- 172.21.15.5:0/4111907481 shutdown_connections 2023-11-24T06:17:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.459+0000 7f01f152e640 1 -- 172.21.15.5:0/4111907481 wait complete. 2023-11-24T06:17:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.461+0000 7f01f152e640 1 Processor -- start 2023-11-24T06:17:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.461+0000 7f01f152e640 1 -- start start 2023-11-24T06:17:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.462+0000 7f01f152e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01ec0d0fa0 0x7f01ec15a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.462+0000 7f01f152e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01ec0d1bc0 0x7f01ec0cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.462+0000 7f01f152e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ec15d3b0 0x7f01ec15fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.462+0000 7f01f152e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec15af10 con 0x7f01ec15d3b0 2023-11-24T06:17:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.462+0000 7f01f152e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0cb340 con 0x7f01ec0d0fa0 2023-11-24T06:17:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.462+0000 7f01f152e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0ca730 con 0x7f01ec0d1bc0 2023-11-24T06:17:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.463+0000 7f01f0d2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ec15d3b0 0x7f01ec15fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.463+0000 7f01f0d2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ec15d3b0 0x7f01ec15fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37248/0 (socket says 172.21.15.5:37248) 2023-11-24T06:17:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.463+0000 7f01f0d2d640 1 -- 172.21.15.5:0/3710712623 learned_addr learned my addr 172.21.15.5:0/3710712623 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.463+0000 7f01eb7fe640 1 --2- 172.21.15.5:0/3710712623 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01ec0d1bc0 0x7f01ec0cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.463+0000 7f01ebfff640 1 --2- 172.21.15.5:0/3710712623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01ec0d0fa0 0x7f01ec15a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.463+0000 7f01f0d2d640 1 -- 172.21.15.5:0/3710712623 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01ec0d1bc0 msgr2=0x7f01ec0cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.463+0000 7f01f0d2d640 1 --2- 172.21.15.5:0/3710712623 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01ec0d1bc0 0x7f01ec0cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.463+0000 7f01f0d2d640 1 -- 172.21.15.5:0/3710712623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01ec0d0fa0 msgr2=0x7f01ec15a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.463+0000 7f01f0d2d640 1 --2- 172.21.15.5:0/3710712623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01ec0d0fa0 0x7f01ec15a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.463+0000 7f01f0d2d640 1 -- 172.21.15.5:0/3710712623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc009b30 con 0x7f01ec15d3b0 2023-11-24T06:17:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.463+0000 7f01f0d2d640 1 --2- 172.21.15.5:0/3710712623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ec15d3b0 0x7f01ec15fa00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f01e400b490 tx=0x7f01e400b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.464+0000 7f01e8ff9640 1 -- 172.21.15.5:0/3710712623 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4011830 con 0x7f01ec15d3b0 2023-11-24T06:17:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.464+0000 7f01e8ff9640 1 -- 172.21.15.5:0/3710712623 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e40119b0 con 0x7f01ec15d3b0 2023-11-24T06:17:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.464+0000 7f01e8ff9640 1 -- 172.21.15.5:0/3710712623 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4011d60 con 0x7f01ec15d3b0 2023-11-24T06:17:38.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.464+0000 7f01f152e640 1 -- 172.21.15.5:0/3710712623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ec0caa10 con 0x7f01ec15d3b0 2023-11-24T06:17:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.464+0000 7f01f152e640 1 -- 172.21.15.5:0/3710712623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec0cafd0 con 0x7f01ec15d3b0 2023-11-24T06:17:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.465+0000 7f01f152e640 1 -- 172.21.15.5:0/3710712623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec0cb4b0 con 0x7f01ec15d3b0 2023-11-24T06:17:38.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.466+0000 7f01e8ff9640 1 -- 172.21.15.5:0/3710712623 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f01e401a400 con 0x7f01ec15d3b0 2023-11-24T06:17:38.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.467+0000 7f01e8ff9640 1 --2- 172.21.15.5:0/3710712623 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f01cc06aed0 0x7f01cc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.467+0000 7f01e8ff9640 1 -- 172.21.15.5:0/3710712623 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01e4089840 con 0x7f01ec15d3b0 2023-11-24T06:17:38.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.468+0000 7f01ebfff640 1 --2- 172.21.15.5:0/3710712623 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f01cc06aed0 0x7f01cc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.468+0000 7f01ebfff640 1 --2- 172.21.15.5:0/3710712623 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f01cc06aed0 0x7f01cc06d390 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01dc035000 tx=0x7f01dc030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:38.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.472+0000 7f01e8ff9640 1 -- 172.21.15.5:0/3710712623 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01e4054800 con 0x7f01ec15d3b0 2023-11-24T06:17:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.673+0000 7f01f152e640 1 -- 172.21.15.5:0/3710712623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f01ec04e840 con 0x7f01ec15d3b0 2023-11-24T06:17:38.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.674+0000 7f01e8ff9640 1 -- 172.21.15.5:0/3710712623 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f01e4057e20 con 0x7f01ec15d3b0 2023-11-24T06:17:38.677 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:17:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.677+0000 7f01f152e640 1 -- 172.21.15.5:0/3710712623 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f01cc06aed0 msgr2=0x7f01cc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.677+0000 7f01f152e640 1 --2- 172.21.15.5:0/3710712623 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f01cc06aed0 0x7f01cc06d390 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f01dc035000 tx=0x7f01dc030000 comp rx=0 tx=0).stop 2023-11-24T06:17:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.677+0000 7f01f152e640 1 -- 172.21.15.5:0/3710712623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ec15d3b0 msgr2=0x7f01ec15fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.677+0000 7f01f152e640 1 --2- 172.21.15.5:0/3710712623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ec15d3b0 0x7f01ec15fa00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f01e400b490 tx=0x7f01e400b950 comp rx=0 tx=0).stop 2023-11-24T06:17:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.677+0000 7f01f152e640 1 -- 172.21.15.5:0/3710712623 shutdown_connections 2023-11-24T06:17:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.677+0000 7f01f152e640 1 --2- 172.21.15.5:0/3710712623 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f01cc06aed0 0x7f01cc06d390 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.677+0000 7f01f152e640 1 --2- 172.21.15.5:0/3710712623 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f01ec0d1bc0 0x7f01ec0cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.677+0000 7f01f152e640 1 --2- 172.21.15.5:0/3710712623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ec15d3b0 0x7f01ec15fa00 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.677+0000 7f01f152e640 1 --2- 172.21.15.5:0/3710712623 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f01ec0d0fa0 0x7f01ec15a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.677+0000 7f01f152e640 1 -- 172.21.15.5:0/3710712623 >> 172.21.15.5:0/3710712623 conn(0x7f01ec0d30a0 msgr2=0x7f01ec0b7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:38.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.677+0000 7f01f152e640 1 -- 172.21.15.5:0/3710712623 shutdown_connections 2023-11-24T06:17:38.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.677+0000 7f01f152e640 1 -- 172.21.15.5:0/3710712623 wait complete. 2023-11-24T06:17:38.698 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T06:17:38.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.923+0000 7f56cf66b640 1 Processor -- start 2023-11-24T06:17:38.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.923+0000 7f56cf66b640 1 -- start start 2023-11-24T06:17:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.925+0000 7f56cf66b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f56c8146c70 0x7f56c806c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.925+0000 7f56cf66b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56c815e6b0 0x7f56c80b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.925+0000 7f56cf66b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f56c80b4860 0x7f56c80b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.925+0000 7f56cf66b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c80b4fc0 con 0x7f56c815e6b0 2023-11-24T06:17:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.925+0000 7f56cf66b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c8147050 con 0x7f56c80b4860 2023-11-24T06:17:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.925+0000 7f56cf66b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c806c210 con 0x7f56c8146c70 2023-11-24T06:17:38.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.925+0000 7f56ccc06640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56c815e6b0 0x7f56c80b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.925+0000 7f56ccc06640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f56c815e6b0 0x7f56c80b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37252/0 (socket says 172.21.15.5:37252) 2023-11-24T06:17:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.925+0000 7f56ccc06640 1 -- 172.21.15.5:0/4278131226 learned_addr learned my addr 172.21.15.5:0/4278131226 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.925+0000 7f56cd407640 1 --2- 172.21.15.5:0/4278131226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f56c8146c70 0x7f56c806c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.925+0000 7f56cdc08640 1 --2- 172.21.15.5:0/4278131226 >> v2:172.21.15.6:3300/0 conn(0x7f56c80b4860 0x7f56c80b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.925+0000 7f56ccc06640 1 -- 172.21.15.5:0/4278131226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f56c8146c70 msgr2=0x7f56c806c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.926+0000 7f56ccc06640 1 --2- 172.21.15.5:0/4278131226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f56c8146c70 0x7f56c806c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.926+0000 7f56ccc06640 1 -- 172.21.15.5:0/4278131226 >> v2:172.21.15.6:3300/0 conn(0x7f56c80b4860 msgr2=0x7f56c80b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.926+0000 7f56ccc06640 1 --2- 172.21.15.5:0/4278131226 >> v2:172.21.15.6:3300/0 conn(0x7f56c80b4860 0x7f56c80b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.926+0000 7f56ccc06640 1 -- 172.21.15.5:0/4278131226 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c80b4ce0 con 0x7f56c815e6b0 2023-11-24T06:17:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.926+0000 7f56cdc08640 1 --2- 172.21.15.5:0/4278131226 >> v2:172.21.15.6:3300/0 conn(0x7f56c80b4860 0x7f56c80b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.927+0000 7f56ccc06640 1 --2- 172.21.15.5:0/4278131226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c815e6b0 0x7f56c80b4320 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f56c8070d60 tx=0x7f56b802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdebf681455ca800 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:38.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.927+0000 7f56bffff640 1 -- 172.21.15.5:0/4278131226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8031860 con 0x7f56c815e6b0 2023-11-24T06:17:38.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.927+0000 7f56bffff640 1 -- 172.21.15.5:0/4278131226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b80319c0 con 0x7f56c815e6b0 2023-11-24T06:17:38.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.927+0000 7f56bffff640 1 -- 172.21.15.5:0/4278131226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8031d50 con 0x7f56c815e6b0 2023-11-24T06:17:38.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.928+0000 7f56cf66b640 1 -- 172.21.15.5:0/4278131226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c815e6b0 msgr2=0x7f56c80b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.928+0000 7f56cf66b640 1 --2- 172.21.15.5:0/4278131226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c815e6b0 0x7f56c80b4320 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f56c8070d60 tx=0x7f56b802b900 comp rx=0 tx=0).stop 2023-11-24T06:17:38.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.928+0000 7f56cf66b640 1 -- 172.21.15.5:0/4278131226 shutdown_connections 2023-11-24T06:17:38.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.928+0000 7f56cf66b640 1 --2- 172.21.15.5:0/4278131226 >> v2:172.21.15.6:3300/0 conn(0x7f56c80b4860 0x7f56c80b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.928+0000 7f56cf66b640 1 --2- 172.21.15.5:0/4278131226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c815e6b0 0x7f56c80b4320 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.928+0000 7f56cf66b640 1 --2- 172.21.15.5:0/4278131226 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f56c8146c70 0x7f56c806c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.928+0000 7f56cf66b640 1 -- 172.21.15.5:0/4278131226 >> 172.21.15.5:0/4278131226 conn(0x7f56c80663d0 msgr2=0x7f56c80667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:38.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.928+0000 7f56cf66b640 1 -- 172.21.15.5:0/4278131226 shutdown_connections 2023-11-24T06:17:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.928+0000 7f56cf66b640 1 -- 172.21.15.5:0/4278131226 wait complete. 2023-11-24T06:17:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.930+0000 7f56cf66b640 1 Processor -- start 2023-11-24T06:17:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.930+0000 7f56cf66b640 1 -- start start 2023-11-24T06:17:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.931+0000 7f56cf66b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f56c80b4860 0x7f56c813ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.931+0000 7f56cf66b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c8146c70 0x7f56c814e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.931+0000 7f56cf66b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56c815e6b0 0x7f56c814ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.931+0000 7f56cf66b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c814feb0 con 0x7f56c8146c70 2023-11-24T06:17:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.931+0000 7f56cf66b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c814ef60 con 0x7f56c815e6b0 2023-11-24T06:17:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.931+0000 7f56cf66b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c814f230 con 0x7f56c80b4860 2023-11-24T06:17:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.931+0000 7f56ccc06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c8146c70 0x7f56c814e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.931+0000 7f56ccc06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c8146c70 0x7f56c814e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37254/0 (socket says 172.21.15.5:37254) 2023-11-24T06:17:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.932+0000 7f56ccc06640 1 -- 172.21.15.5:0/697239314 learned_addr learned my addr 172.21.15.5:0/697239314 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.932+0000 7f56cd407640 1 --2- 172.21.15.5:0/697239314 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f56c80b4860 0x7f56c813ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.932+0000 7f56ccc06640 1 -- 172.21.15.5:0/697239314 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f56c80b4860 msgr2=0x7f56c813ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.932+0000 7f56ccc06640 1 --2- 172.21.15.5:0/697239314 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f56c80b4860 0x7f56c813ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.932+0000 7f56cdc08640 1 --2- 172.21.15.5:0/697239314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56c815e6b0 0x7f56c814ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.932+0000 7f56ccc06640 1 -- 172.21.15.5:0/697239314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56c815e6b0 msgr2=0x7f56c814ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.932+0000 7f56ccc06640 1 --2- 172.21.15.5:0/697239314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56c815e6b0 0x7f56c814ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.932+0000 7f56ccc06640 1 -- 172.21.15.5:0/697239314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b80094d0 con 0x7f56c8146c70 2023-11-24T06:17:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.932+0000 7f56ccc06640 1 --2- 172.21.15.5:0/697239314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c8146c70 0x7f56c814e4e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f56b802b940 tx=0x7f56b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.933+0000 7f56bdffb640 1 -- 172.21.15.5:0/697239314 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8031850 con 0x7f56c8146c70 2023-11-24T06:17:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.933+0000 7f56bdffb640 1 -- 172.21.15.5:0/697239314 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b80443f0 con 0x7f56c8146c70 2023-11-24T06:17:38.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.933+0000 7f56bdffb640 1 -- 172.21.15.5:0/697239314 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80447a0 con 0x7f56c8146c70 2023-11-24T06:17:38.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.933+0000 7f56cf66b640 1 -- 172.21.15.5:0/697239314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c814f510 con 0x7f56c8146c70 2023-11-24T06:17:38.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.933+0000 7f56cf66b640 1 -- 172.21.15.5:0/697239314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c814fad0 con 0x7f56c8146c70 2023-11-24T06:17:38.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.934+0000 7f56cf66b640 1 -- 172.21.15.5:0/697239314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c8074770 con 0x7f56c8146c70 2023-11-24T06:17:38.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.935+0000 7f56bdffb640 1 -- 172.21.15.5:0/697239314 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f56b804d450 con 0x7f56c8146c70 2023-11-24T06:17:38.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.936+0000 7f56bdffb640 1 --2- 172.21.15.5:0/697239314 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f56ac06aed0 0x7f56ac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:38.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.936+0000 7f56bdffb640 1 -- 172.21.15.5:0/697239314 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56b80abfe0 con 0x7f56c8146c70 2023-11-24T06:17:38.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.936+0000 7f56cd407640 1 --2- 172.21.15.5:0/697239314 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f56ac06aed0 0x7f56ac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:38.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.937+0000 7f56cd407640 1 --2- 172.21.15.5:0/697239314 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f56ac06aed0 0x7f56ac06d390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f56c4006fd0 tx=0x7f56c4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:38.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:38.941+0000 7f56bdffb640 1 -- 172.21.15.5:0/697239314 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56b80dee90 con 0x7f56c8146c70 2023-11-24T06:17:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.150+0000 7f56cf66b640 1 -- 172.21.15.5:0/697239314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f56c8070e10 con 0x7f56c8146c70 2023-11-24T06:17:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.151+0000 7f56bdffb640 1 -- 172.21.15.5:0/697239314 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7f56b8035070 con 0x7f56c8146c70 2023-11-24T06:17:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2023-11-24T06:17:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.153+0000 7f56cf66b640 1 -- 172.21.15.5:0/697239314 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f56ac06aed0 msgr2=0x7f56ac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.153+0000 7f56cf66b640 1 --2- 172.21.15.5:0/697239314 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f56ac06aed0 0x7f56ac06d390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f56c4006fd0 tx=0x7f56c4008040 comp rx=0 tx=0).stop 2023-11-24T06:17:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.153+0000 7f56cf66b640 1 -- 172.21.15.5:0/697239314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c8146c70 msgr2=0x7f56c814e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.153+0000 7f56cf66b640 1 --2- 172.21.15.5:0/697239314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c8146c70 0x7f56c814e4e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f56b802b940 tx=0x7f56b80046c0 comp rx=0 tx=0).stop 2023-11-24T06:17:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.153+0000 7f56cf66b640 1 -- 172.21.15.5:0/697239314 shutdown_connections 2023-11-24T06:17:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.153+0000 7f56cf66b640 1 --2- 172.21.15.5:0/697239314 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f56ac06aed0 0x7f56ac06d390 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.153+0000 7f56cf66b640 1 --2- 172.21.15.5:0/697239314 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f56c815e6b0 0x7f56c814ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.153+0000 7f56cf66b640 1 --2- 172.21.15.5:0/697239314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56c8146c70 0x7f56c814e4e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.153+0000 7f56cf66b640 1 --2- 172.21.15.5:0/697239314 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f56c80b4860 0x7f56c813ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.153+0000 7f56cf66b640 1 -- 172.21.15.5:0/697239314 >> 172.21.15.5:0/697239314 conn(0x7f56c80663d0 msgr2=0x7f56c80ca370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.154+0000 7f56cf66b640 1 -- 172.21.15.5:0/697239314 shutdown_connections 2023-11-24T06:17:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.154+0000 7f56cf66b640 1 -- 172.21.15.5:0/697239314 wait complete. 2023-11-24T06:17:39.176 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi006.front.sepia.ceph.com... 2023-11-24T06:17:39.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:39.177 DEBUG:teuthology.orchestra.run.smithi006:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-24T06:17:39.273 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi006.stderr:ceph-fuse[94670]: fuse finished with error 0 and tester_r 0 2023-11-24T06:17:39.278 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T06:17:39.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:39.278 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T06:17:39.461 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:17:39.461 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:17:39.526 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:17:39.526 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T06:17:39.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:39.526 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T06:17:39.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:17:39.585 INFO:teuthology.orchestra.run.smithi005.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-24T06:17:39.586 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:17:39.586 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:17:39.651 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:17:39.651 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T06:17:39.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:39.652 DEBUG:teuthology.orchestra.run.smithi006:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T06:17:39.713 INFO:teuthology.orchestra.run.smithi006.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-24T06:17:39.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:17:39.714 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:17:39.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.944+0000 7fc93f0f1640 1 Processor -- start 2023-11-24T06:17:39.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.945+0000 7fc93f0f1640 1 -- start start 2023-11-24T06:17:39.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.946+0000 7fc93f0f1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc93815d5b0 0x7fc9380d3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:39.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.946+0000 7fc93f0f1640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fc9380b1650 0x7fc9380d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:39.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.946+0000 7fc93f0f1640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9380cf0c0 con 0x7fc9380d14a0 2023-11-24T06:17:39.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.946+0000 7fc93f0f1640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9380b01b0 con 0x7fc93815d5b0 2023-11-24T06:17:39.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.946+0000 7fc93f0f1640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc93815d990 con 0x7fc9380b1650 2023-11-24T06:17:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.947+0000 7fc93e0ef640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc93815d5b0 0x7fc9380d3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.947+0000 7fc93e0ef640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc93815d5b0 0x7fc9380d3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37264/0 (socket says 172.21.15.5:37264) 2023-11-24T06:17:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.947+0000 7fc93e0ef640 1 -- 172.21.15.5:0/3656536427 learned_addr learned my addr 172.21.15.5:0/3656536427 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.947+0000 7fc93e8f0640 1 --2- 172.21.15.5:0/3656536427 >> v2:172.21.15.6:3300/0 conn(0x7fc9380b1650 0x7fc9380d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.947+0000 7fc93e0ef640 1 -- 172.21.15.5:0/3656536427 >> v2:172.21.15.6:3300/0 conn(0x7fc9380b1650 msgr2=0x7fc9380d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.947+0000 7fc93e0ef640 1 --2- 172.21.15.5:0/3656536427 >> v2:172.21.15.6:3300/0 conn(0x7fc9380b1650 0x7fc9380d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.947+0000 7fc93e0ef640 1 -- 172.21.15.5:0/3656536427 >> v1:172.21.15.6:6789/0 conn(0x7fc9380d14a0 legacy=0x7fc9380cf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.947+0000 7fc93e0ef640 1 -- 172.21.15.5:0/3656536427 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9380d1e40 con 0x7fc93815d5b0 2023-11-24T06:17:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.947+0000 7fc93e8f0640 1 --2- 172.21.15.5:0/3656536427 >> v2:172.21.15.6:3300/0 conn(0x7fc9380b1650 0x7fc9380d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.949+0000 7fc93e0ef640 1 --2- 172.21.15.5:0/3656536427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc93815d5b0 0x7fc9380d3990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc93808bc30 tx=0x7fc93402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fa544d7b815962 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:39.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.949+0000 7fc93d0ed640 1 -- 172.21.15.5:0/3656536427 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934031870 con 0x7fc93815d5b0 2023-11-24T06:17:39.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.949+0000 7fc93d0ed640 1 -- 172.21.15.5:0/3656536427 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9340319d0 con 0x7fc93815d5b0 2023-11-24T06:17:39.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.949+0000 7fc93d0ed640 1 -- 172.21.15.5:0/3656536427 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934031d60 con 0x7fc93815d5b0 2023-11-24T06:17:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.950+0000 7fc93f0f1640 1 -- 172.21.15.5:0/3656536427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc93815d5b0 msgr2=0x7fc9380d3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.950+0000 7fc93f0f1640 1 --2- 172.21.15.5:0/3656536427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc93815d5b0 0x7fc9380d3990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fc93808bc30 tx=0x7fc93402b910 comp rx=0 tx=0).stop 2023-11-24T06:17:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.950+0000 7fc93f0f1640 1 -- 172.21.15.5:0/3656536427 shutdown_connections 2023-11-24T06:17:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.950+0000 7fc93f0f1640 1 --2- 172.21.15.5:0/3656536427 >> v2:172.21.15.6:3300/0 conn(0x7fc9380b1650 0x7fc9380d1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.950+0000 7fc93f0f1640 1 --2- 172.21.15.5:0/3656536427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc93815d5b0 0x7fc9380d3990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.950+0000 7fc93f0f1640 1 -- 172.21.15.5:0/3656536427 >> 172.21.15.5:0/3656536427 conn(0x7fc93809c3b0 msgr2=0x7fc9380d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.950+0000 7fc93f0f1640 1 -- 172.21.15.5:0/3656536427 shutdown_connections 2023-11-24T06:17:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.950+0000 7fc93f0f1640 1 -- 172.21.15.5:0/3656536427 wait complete. 2023-11-24T06:17:39.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.952+0000 7fc93f0f1640 1 Processor -- start 2023-11-24T06:17:39.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.952+0000 7fc93f0f1640 1 -- start start 2023-11-24T06:17:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.953+0000 7fc93f0f1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9380b1650 0x7fc93814e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.953+0000 7fc93f0f1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9380d14a0 0x7fc9380c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.953+0000 7fc93f0f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9380c9860 0x7fc93814ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.953+0000 7fc93f0f1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380c8030 con 0x7fc9380c9860 2023-11-24T06:17:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.953+0000 7fc93f0f1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93814ea00 con 0x7fc9380d14a0 2023-11-24T06:17:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.953+0000 7fc93f0f1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380c8280 con 0x7fc9380b1650 2023-11-24T06:17:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.953+0000 7fc93e8f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9380c9860 0x7fc93814ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.953+0000 7fc93e8f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9380c9860 0x7fc93814ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37274/0 (socket says 172.21.15.5:37274) 2023-11-24T06:17:39.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.953+0000 7fc93e8f0640 1 -- 172.21.15.5:0/2043385330 learned_addr learned my addr 172.21.15.5:0/2043385330 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.954+0000 7fc93e0ef640 1 --2- 172.21.15.5:0/2043385330 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9380b1650 0x7fc93814e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.954+0000 7fc93d8ee640 1 --2- 172.21.15.5:0/2043385330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9380d14a0 0x7fc9380c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.954+0000 7fc93e8f0640 1 -- 172.21.15.5:0/2043385330 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9380b1650 msgr2=0x7fc93814e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.954+0000 7fc93e8f0640 1 --2- 172.21.15.5:0/2043385330 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9380b1650 0x7fc93814e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.954+0000 7fc93e8f0640 1 -- 172.21.15.5:0/2043385330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9380d14a0 msgr2=0x7fc9380c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.954+0000 7fc93e8f0640 1 --2- 172.21.15.5:0/2043385330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9380d14a0 0x7fc9380c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.954+0000 7fc93e8f0640 1 -- 172.21.15.5:0/2043385330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9340094d0 con 0x7fc9380c9860 2023-11-24T06:17:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.954+0000 7fc93e8f0640 1 --2- 172.21.15.5:0/2043385330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9380c9860 0x7fc93814ef10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc93000c610 tx=0x7fc93000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.955+0000 7fc92effd640 1 -- 172.21.15.5:0/2043385330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc930004220 con 0x7fc9380c9860 2023-11-24T06:17:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.955+0000 7fc92effd640 1 -- 172.21.15.5:0/2043385330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9300043a0 con 0x7fc9380c9860 2023-11-24T06:17:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.955+0000 7fc92effd640 1 -- 172.21.15.5:0/2043385330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc930003850 con 0x7fc9380c9860 2023-11-24T06:17:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.955+0000 7fc93f0f1640 1 -- 172.21.15.5:0/2043385330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9380c8560 con 0x7fc9380c9860 2023-11-24T06:17:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.955+0000 7fc93f0f1640 1 -- 172.21.15.5:0/2043385330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9380c8b20 con 0x7fc9380c9860 2023-11-24T06:17:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.956+0000 7fc93f0f1640 1 -- 172.21.15.5:0/2043385330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc93804e840 con 0x7fc9380c9860 2023-11-24T06:17:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.957+0000 7fc92effd640 1 -- 172.21.15.5:0/2043385330 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc9300039b0 con 0x7fc9380c9860 2023-11-24T06:17:39.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.958+0000 7fc92effd640 1 --2- 172.21.15.5:0/2043385330 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc91c06aed0 0x7fc91c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.958+0000 7fc92effd640 1 -- 172.21.15.5:0/2043385330 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9300895d0 con 0x7fc9380c9860 2023-11-24T06:17:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.958+0000 7fc93e0ef640 1 --2- 172.21.15.5:0/2043385330 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc91c06aed0 0x7fc91c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.959+0000 7fc93e0ef640 1 --2- 172.21.15.5:0/2043385330 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc91c06aed0 0x7fc91c06d390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc934036000 tx=0x7fc93403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:39.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:39.962+0000 7fc92effd640 1 -- 172.21.15.5:0/2043385330 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc930054480 con 0x7fc9380c9860 2023-11-24T06:17:40.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.196+0000 7fc93f0f1640 1 -- 172.21.15.5:0/2043385330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9380678d0 con 0x7fc9380c9860 2023-11-24T06:17:40.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.198+0000 7fc92effd640 1 -- 172.21.15.5:0/2043385330 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4901 (secure 0 0 0) 0x7fc930057aa0 con 0x7fc9380c9860 2023-11-24T06:17:40.201 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:40.201 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6833/2072679798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2072679798},{"type":"v1","addr":"172.21.15.6:6833","nonce":2072679798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3855353235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3855353235},{"type":"v1","addr":"172.21.15.6:6835","nonce":3855353235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2394562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2394562290},{"type":"v1","addr":"172.21.15.5:6836","nonce":2394562290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:17:16.874899+0000","modified":"2023-11-24T06:17:20.326601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5217":{"gid":5217,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/3690265514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3690265514},{"type":"v1","addr":"172.21.15.5:6837","nonce":3690265514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:17:40.201 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 18 2023-11-24T06:17:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.200+0000 7fc93f0f1640 1 -- 172.21.15.5:0/2043385330 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc91c06aed0 msgr2=0x7fc91c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.200+0000 7fc93f0f1640 1 --2- 172.21.15.5:0/2043385330 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc91c06aed0 0x7fc91c06d390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc934036000 tx=0x7fc93403c040 comp rx=0 tx=0).stop 2023-11-24T06:17:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.200+0000 7fc93f0f1640 1 -- 172.21.15.5:0/2043385330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9380c9860 msgr2=0x7fc93814ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.200+0000 7fc93f0f1640 1 --2- 172.21.15.5:0/2043385330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9380c9860 0x7fc93814ef10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc93000c610 tx=0x7fc93000cad0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.200+0000 7fc93f0f1640 1 -- 172.21.15.5:0/2043385330 shutdown_connections 2023-11-24T06:17:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.200+0000 7fc93f0f1640 1 --2- 172.21.15.5:0/2043385330 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fc91c06aed0 0x7fc91c06d390 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.200+0000 7fc93f0f1640 1 --2- 172.21.15.5:0/2043385330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9380c9860 0x7fc93814ef10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.200+0000 7fc93f0f1640 1 --2- 172.21.15.5:0/2043385330 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fc9380d14a0 0x7fc9380c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.200+0000 7fc93f0f1640 1 --2- 172.21.15.5:0/2043385330 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fc9380b1650 0x7fc93814e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.200+0000 7fc93f0f1640 1 -- 172.21.15.5:0/2043385330 >> 172.21.15.5:0/2043385330 conn(0x7fc93809c3b0 msgr2=0x7fc93815dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.200+0000 7fc93f0f1640 1 -- 172.21.15.5:0/2043385330 shutdown_connections 2023-11-24T06:17:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.201+0000 7fc93f0f1640 1 -- 172.21.15.5:0/2043385330 wait complete. 2023-11-24T06:17:40.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:17:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.445+0000 7f5773368640 1 Processor -- start 2023-11-24T06:17:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.445+0000 7f5773368640 1 -- start start 2023-11-24T06:17:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.447+0000 7f5773368640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f576c15a130 0x7f576c0b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.447+0000 7f5773368640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f576c15dda0 0x7f576c0aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.447+0000 7f5773368640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576c0cf760 con 0x7f576c0ccc00 2023-11-24T06:17:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.447+0000 7f5773368640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576c06e560 con 0x7f576c15dda0 2023-11-24T06:17:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.447+0000 7f5773368640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576c15e260 con 0x7f576c15a130 2023-11-24T06:17:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.448+0000 7f5770903640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f576c15dda0 0x7f576c0aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.448+0000 7f5770903640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f576c15dda0 0x7f576c0aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37282/0 (socket says 172.21.15.5:37282) 2023-11-24T06:17:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.448+0000 7f5770903640 1 -- 172.21.15.5:0/599112650 learned_addr learned my addr 172.21.15.5:0/599112650 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.448+0000 7f5771104640 1 --2- 172.21.15.5:0/599112650 >> v2:172.21.15.6:3300/0 conn(0x7f576c15a130 0x7f576c0b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.448+0000 7f5770903640 1 -- 172.21.15.5:0/599112650 >> v2:172.21.15.6:3300/0 conn(0x7f576c15a130 msgr2=0x7f576c0b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.448+0000 7f5770903640 1 --2- 172.21.15.5:0/599112650 >> v2:172.21.15.6:3300/0 conn(0x7f576c15a130 0x7f576c0b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.448+0000 7f5770903640 1 -- 172.21.15.5:0/599112650 >> v1:172.21.15.6:6789/0 conn(0x7f576c0ccc00 legacy=0x7f576c06e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.448+0000 7f5770903640 1 -- 172.21.15.5:0/599112650 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c15e540 con 0x7f576c15dda0 2023-11-24T06:17:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.449+0000 7f5770903640 1 --2- 172.21.15.5:0/599112650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f576c15dda0 0x7f576c0aee00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f576c06b800 tx=0x7f575c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0a6464e261740e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:40.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.450+0000 7f5763fff640 1 -- 172.21.15.5:0/599112650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c0317e0 con 0x7f576c15dda0 2023-11-24T06:17:40.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.450+0000 7f5763fff640 1 -- 172.21.15.5:0/599112650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c031940 con 0x7f576c15dda0 2023-11-24T06:17:40.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.450+0000 7f5763fff640 1 -- 172.21.15.5:0/599112650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c031cd0 con 0x7f576c15dda0 2023-11-24T06:17:40.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.451+0000 7f5773368640 1 -- 172.21.15.5:0/599112650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f576c15dda0 msgr2=0x7f576c0aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:40.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.451+0000 7f5773368640 1 --2- 172.21.15.5:0/599112650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f576c15dda0 0x7f576c0aee00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f576c06b800 tx=0x7f575c02b880 comp rx=0 tx=0).stop 2023-11-24T06:17:40.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.451+0000 7f5773368640 1 -- 172.21.15.5:0/599112650 shutdown_connections 2023-11-24T06:17:40.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.451+0000 7f5773368640 1 --2- 172.21.15.5:0/599112650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f576c15dda0 0x7f576c0aee00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.451+0000 7f5773368640 1 --2- 172.21.15.5:0/599112650 >> v2:172.21.15.6:3300/0 conn(0x7f576c15a130 0x7f576c0b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.451+0000 7f5773368640 1 -- 172.21.15.5:0/599112650 >> 172.21.15.5:0/599112650 conn(0x7f576c0b23a0 msgr2=0x7f576c0cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:40.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.451+0000 7f5773368640 1 -- 172.21.15.5:0/599112650 shutdown_connections 2023-11-24T06:17:40.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.451+0000 7f5773368640 1 -- 172.21.15.5:0/599112650 wait complete. 2023-11-24T06:17:40.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.453+0000 7f5773368640 1 Processor -- start 2023-11-24T06:17:40.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.453+0000 7f5773368640 1 -- start start 2023-11-24T06:17:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.454+0000 7f5773368640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f576c0ccc00 0x7f576c154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.454+0000 7f5773368640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f576c15a130 0x7f576c151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:40.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.454+0000 7f5773368640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f576c15dda0 0x7f576c14c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.454+0000 7f5773368640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0ac870 con 0x7f576c15dda0 2023-11-24T06:17:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.454+0000 7f5773368640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c151550 con 0x7f576c0ccc00 2023-11-24T06:17:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.454+0000 7f5773368640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0acac0 con 0x7f576c15a130 2023-11-24T06:17:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.454+0000 7f5771905640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f576c15dda0 0x7f576c14c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.454+0000 7f5771905640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f576c15dda0 0x7f576c14c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37286/0 (socket says 172.21.15.5:37286) 2023-11-24T06:17:40.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.454+0000 7f5771905640 1 -- 172.21.15.5:0/3056211827 learned_addr learned my addr 172.21.15.5:0/3056211827 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:40.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.455+0000 7f5770903640 1 --2- 172.21.15.5:0/3056211827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f576c15a130 0x7f576c151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:40.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.455+0000 7f5771104640 1 --2- 172.21.15.5:0/3056211827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f576c0ccc00 0x7f576c154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:40.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.455+0000 7f5771905640 1 -- 172.21.15.5:0/3056211827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f576c15a130 msgr2=0x7f576c151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:40.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.455+0000 7f5771905640 1 --2- 172.21.15.5:0/3056211827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f576c15a130 0x7f576c151010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.455+0000 7f5771905640 1 -- 172.21.15.5:0/3056211827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f576c0ccc00 msgr2=0x7f576c154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:40.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.455+0000 7f5771905640 1 --2- 172.21.15.5:0/3056211827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f576c0ccc00 0x7f576c154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.455+0000 7f5771905640 1 -- 172.21.15.5:0/3056211827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c0094d0 con 0x7f576c15dda0 2023-11-24T06:17:40.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.456+0000 7f5771905640 1 --2- 172.21.15.5:0/3056211827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f576c15dda0 0x7f576c14c530 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f576400c780 tx=0x7f576400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:40.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.456+0000 7f5761ffb640 1 -- 172.21.15.5:0/3056211827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5764004b60 con 0x7f576c15dda0 2023-11-24T06:17:40.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.456+0000 7f5761ffb640 1 -- 172.21.15.5:0/3056211827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576400ce90 con 0x7f576c15dda0 2023-11-24T06:17:40.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.456+0000 7f5761ffb640 1 -- 172.21.15.5:0/3056211827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576400a850 con 0x7f576c15dda0 2023-11-24T06:17:40.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.456+0000 7f5773368640 1 -- 172.21.15.5:0/3056211827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576c15a690 con 0x7f576c15dda0 2023-11-24T06:17:40.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.456+0000 7f5773368640 1 -- 172.21.15.5:0/3056211827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576c0ad080 con 0x7f576c15dda0 2023-11-24T06:17:40.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.457+0000 7f5773368640 1 -- 172.21.15.5:0/3056211827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576c074770 con 0x7f576c15dda0 2023-11-24T06:17:40.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.458+0000 7f5761ffb640 1 -- 172.21.15.5:0/3056211827 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f576400a9b0 con 0x7f576c15dda0 2023-11-24T06:17:40.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.459+0000 7f5761ffb640 1 --2- 172.21.15.5:0/3056211827 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f575006af00 0x7f575006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:40.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.460+0000 7f5761ffb640 1 -- 172.21.15.5:0/3056211827 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57640898f0 con 0x7f576c15dda0 2023-11-24T06:17:40.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.460+0000 7f5771104640 1 --2- 172.21.15.5:0/3056211827 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f575006af00 0x7f575006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.460+0000 7f5771104640 1 --2- 172.21.15.5:0/3056211827 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f575006af00 0x7f575006d3c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5768006fd0 tx=0x7f5768008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:40.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.464+0000 7f5761ffb640 1 -- 172.21.15.5:0/3056211827 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5764054820 con 0x7f576c15dda0 2023-11-24T06:17:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.661+0000 7f5773368640 1 -- 172.21.15.5:0/3056211827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f576c0af2a0 con 0x7f576c15dda0 2023-11-24T06:17:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.663+0000 7f5761ffb640 1 -- 172.21.15.5:0/3056211827 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16425 (secure 0 0 0) 0x7f5764057e40 con 0x7f576c15dda0 2023-11-24T06:17:40.669 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:40.669 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:17:18.380558+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:17:14.868917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:17:15.895189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:17:40.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.666+0000 7f5773368640 1 -- 172.21.15.5:0/3056211827 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f575006af00 msgr2=0x7f575006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:40.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.666+0000 7f5773368640 1 --2- 172.21.15.5:0/3056211827 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f575006af00 0x7f575006d3c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5768006fd0 tx=0x7f5768008040 comp rx=0 tx=0).stop 2023-11-24T06:17:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.666+0000 7f5773368640 1 -- 172.21.15.5:0/3056211827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f576c15dda0 msgr2=0x7f576c14c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.666+0000 7f5773368640 1 --2- 172.21.15.5:0/3056211827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f576c15dda0 0x7f576c14c530 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f576400c780 tx=0x7f576400cc40 comp rx=0 tx=0).stop 2023-11-24T06:17:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.666+0000 7f5773368640 1 -- 172.21.15.5:0/3056211827 shutdown_connections 2023-11-24T06:17:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.666+0000 7f5773368640 1 --2- 172.21.15.5:0/3056211827 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f575006af00 0x7f575006d3c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.666+0000 7f5773368640 1 --2- 172.21.15.5:0/3056211827 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f576c15a130 0x7f576c151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.666+0000 7f5773368640 1 --2- 172.21.15.5:0/3056211827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f576c15dda0 0x7f576c14c530 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.666+0000 7f5773368640 1 --2- 172.21.15.5:0/3056211827 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f576c0ccc00 0x7f576c154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.666+0000 7f5773368640 1 -- 172.21.15.5:0/3056211827 >> 172.21.15.5:0/3056211827 conn(0x7f576c0b23a0 msgr2=0x7f576c06ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.666+0000 7f5773368640 1 -- 172.21.15.5:0/3056211827 shutdown_connections 2023-11-24T06:17:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.666+0000 7f5773368640 1 -- 172.21.15.5:0/3056211827 wait complete. 2023-11-24T06:17:40.690 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T06:15:40.164863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T06:17:14.868917+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T06:17:15.895189+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T06:17:40.691 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T06:17:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.912+0000 7f2f8e5a5640 1 Processor -- start 2023-11-24T06:17:40.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.912+0000 7f2f8e5a5640 1 -- start start 2023-11-24T06:17:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.914+0000 7f2f8e5a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f880ad450 0x7f2f88154280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.914+0000 7f2f8e5a5640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2f88070850 0x7f2f88071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.914+0000 7f2f8e5a5640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f880ce650 con 0x7f2f88153ea0 2023-11-24T06:17:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.914+0000 7f2f8e5a5640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f881547c0 con 0x7f2f88070850 2023-11-24T06:17:40.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.914+0000 7f2f8e5a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f880ad8b0 con 0x7f2f880ad450 2023-11-24T06:17:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.915+0000 7f2f8dda4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2f88070850 0x7f2f88071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.915+0000 7f2f8cda2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f880ad450 0x7f2f88154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.915+0000 7f2f8cda2640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f880ad450 0x7f2f88154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:60702/0 (socket says 172.21.15.5:60702) 2023-11-24T06:17:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.915+0000 7f2f8cda2640 1 -- 172.21.15.5:0/1429511869 learned_addr learned my addr 172.21.15.5:0/1429511869 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:40.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.915+0000 7f2f8dda4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2f88070850 0x7f2f88071350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:36612/0 (socket says 172.21.15.5:36612) 2023-11-24T06:17:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.916+0000 7f2f8cda2640 1 -- 172.21.15.5:0/1429511869 >> v2:172.21.15.6:3300/0 conn(0x7f2f88070850 msgr2=0x7f2f88071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.916+0000 7f2f8cda2640 1 --2- 172.21.15.5:0/1429511869 >> v2:172.21.15.6:3300/0 conn(0x7f2f88070850 0x7f2f88071350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.916+0000 7f2f8cda2640 1 -- 172.21.15.5:0/1429511869 >> v1:172.21.15.6:6789/0 conn(0x7f2f88153ea0 legacy=0x7f2f8815d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.916+0000 7f2f8cda2640 1 -- 172.21.15.5:0/1429511869 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8815e0a0 con 0x7f2f880ad450 2023-11-24T06:17:40.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.916+0000 7f2f87fff640 1 -- 172.21.15.5:0/1429511869 <== mon.1 v1:172.21.15.6:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2718306675 0 0) 0x7f2f880ce650 con 0x7f2f88153ea0 2023-11-24T06:17:40.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.918+0000 7f2f8cda2640 1 --2- 172.21.15.5:0/1429511869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f880ad450 0x7f2f88154280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009870 tx=0x7f2f7c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8abf3f9df91457a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.918+0000 7f2f87fff640 1 -- 172.21.15.5:0/1429511869 <== mon.2 v2:172.21.15.6:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c030840 con 0x7f2f880ad450 2023-11-24T06:17:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.918+0000 7f2f87fff640 1 -- 172.21.15.5:0/1429511869 <== mon.2 v2:172.21.15.6:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c0309a0 con 0x7f2f880ad450 2023-11-24T06:17:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.918+0000 7f2f87fff640 1 -- 172.21.15.5:0/1429511869 <== mon.2 v2:172.21.15.6:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c030d30 con 0x7f2f880ad450 2023-11-24T06:17:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.919+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/1429511869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f880ad450 msgr2=0x7f2f88154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.919+0000 7f2f8e5a5640 1 --2- 172.21.15.5:0/1429511869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f880ad450 0x7f2f88154280 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c009870 tx=0x7f2f7c02b980 comp rx=0 tx=0).stop 2023-11-24T06:17:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.919+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/1429511869 shutdown_connections 2023-11-24T06:17:40.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.919+0000 7f2f8e5a5640 1 --2- 172.21.15.5:0/1429511869 >> v2:172.21.15.6:3300/0 conn(0x7f2f88070850 0x7f2f88071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.919+0000 7f2f8e5a5640 1 --2- 172.21.15.5:0/1429511869 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f880ad450 0x7f2f88154280 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.919+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/1429511869 >> 172.21.15.5:0/1429511869 conn(0x7f2f880cec10 msgr2=0x7f2f880cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:40.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.919+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/1429511869 shutdown_connections 2023-11-24T06:17:40.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.919+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/1429511869 wait complete. 2023-11-24T06:17:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.921+0000 7f2f8e5a5640 1 Processor -- start 2023-11-24T06:17:40.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.921+0000 7f2f8e5a5640 1 -- start start 2023-11-24T06:17:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.922+0000 7f2f8e5a5640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f88070850 0x7f2f880afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.922+0000 7f2f8e5a5640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f880ad450 0x7f2f88144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:40.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.922+0000 7f2f8e5a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f88153ea0 0x7f2f8814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.922+0000 7f2f8e5a5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f880af9a0 con 0x7f2f88153ea0 2023-11-24T06:17:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.922+0000 7f2f8e5a5640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f88144bd0 con 0x7f2f88070850 2023-11-24T06:17:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.922+0000 7f2f8e5a5640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f880b0d60 con 0x7f2f880ad450 2023-11-24T06:17:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.923+0000 7f2f8dda4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f88153ea0 0x7f2f8814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.923+0000 7f2f8dda4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f88153ea0 0x7f2f8814ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37296/0 (socket says 172.21.15.5:37296) 2023-11-24T06:17:40.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.923+0000 7f2f8dda4640 1 -- 172.21.15.5:0/3295313545 learned_addr learned my addr 172.21.15.5:0/3295313545 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.923+0000 7f2f8cda2640 1 --2- 172.21.15.5:0/3295313545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f880ad450 0x7f2f88144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.923+0000 7f2f8d5a3640 1 --2- 172.21.15.5:0/3295313545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f88070850 0x7f2f880afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.923+0000 7f2f8dda4640 1 -- 172.21.15.5:0/3295313545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f880ad450 msgr2=0x7f2f88144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:40.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.923+0000 7f2f8dda4640 1 --2- 172.21.15.5:0/3295313545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f880ad450 0x7f2f88144690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.923+0000 7f2f8dda4640 1 -- 172.21.15.5:0/3295313545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f88070850 msgr2=0x7f2f880afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.923+0000 7f2f8dda4640 1 --2- 172.21.15.5:0/3295313545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f88070850 0x7f2f880afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.923+0000 7f2f8dda4640 1 -- 172.21.15.5:0/3295313545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c0094d0 con 0x7f2f88153ea0 2023-11-24T06:17:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.924+0000 7f2f8dda4640 1 --2- 172.21.15.5:0/3295313545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f88153ea0 0x7f2f8814ff50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000e7f0 tx=0x7f2f8000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.924+0000 7f2f85ffb640 1 -- 172.21.15.5:0/3295313545 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f80009510 con 0x7f2f88153ea0 2023-11-24T06:17:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.924+0000 7f2f85ffb640 1 -- 172.21.15.5:0/3295313545 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8000ee40 con 0x7f2f88153ea0 2023-11-24T06:17:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.924+0000 7f2f85ffb640 1 -- 172.21.15.5:0/3295313545 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8000cb80 con 0x7f2f88153ea0 2023-11-24T06:17:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.924+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/3295313545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f880b1040 con 0x7f2f88153ea0 2023-11-24T06:17:40.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.924+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/3295313545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f880af2e0 con 0x7f2f88153ea0 2023-11-24T06:17:40.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.925+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/3295313545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f880afae0 con 0x7f2f88153ea0 2023-11-24T06:17:40.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.926+0000 7f2f85ffb640 1 -- 172.21.15.5:0/3295313545 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f2f8000cce0 con 0x7f2f88153ea0 2023-11-24T06:17:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.927+0000 7f2f85ffb640 1 --2- 172.21.15.5:0/3295313545 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f7406aed0 0x7f2f7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.927+0000 7f2f85ffb640 1 -- 172.21.15.5:0/3295313545 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f800897a0 con 0x7f2f88153ea0 2023-11-24T06:17:40.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.927+0000 7f2f8d5a3640 1 --2- 172.21.15.5:0/3295313545 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f7406aed0 0x7f2f7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:40.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.928+0000 7f2f8d5a3640 1 --2- 172.21.15.5:0/3295313545 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f7406aed0 0x7f2f7406d390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2f78007fd0 tx=0x7f2f78009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:40.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:40.931+0000 7f2f85ffb640 1 -- 172.21.15.5:0/3295313545 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f880afae0 con 0x7f2f88153ea0 2023-11-24T06:17:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.125+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/3295313545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f880afae0 con 0x7f2f88153ea0 2023-11-24T06:17:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.126+0000 7f2f85ffb640 1 -- 172.21.15.5:0/3295313545 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f880afae0 con 0x7f2f88153ea0 2023-11-24T06:17:41.129 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-11-24T06:17:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.130+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/3295313545 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f7406aed0 msgr2=0x7f2f7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.130+0000 7f2f8e5a5640 1 --2- 172.21.15.5:0/3295313545 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f7406aed0 0x7f2f7406d390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2f78007fd0 tx=0x7f2f78009040 comp rx=0 tx=0).stop 2023-11-24T06:17:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.130+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/3295313545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f88153ea0 msgr2=0x7f2f8814ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.130+0000 7f2f8e5a5640 1 --2- 172.21.15.5:0/3295313545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f88153ea0 0x7f2f8814ff50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000e7f0 tx=0x7f2f8000ebf0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.130+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/3295313545 shutdown_connections 2023-11-24T06:17:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.130+0000 7f2f8e5a5640 1 --2- 172.21.15.5:0/3295313545 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f7406aed0 0x7f2f7406d390 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.130+0000 7f2f8e5a5640 1 --2- 172.21.15.5:0/3295313545 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f880ad450 0x7f2f88144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.130+0000 7f2f8e5a5640 1 --2- 172.21.15.5:0/3295313545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f88153ea0 0x7f2f8814ff50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.130+0000 7f2f8e5a5640 1 --2- 172.21.15.5:0/3295313545 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f88070850 0x7f2f880afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.130+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/3295313545 >> 172.21.15.5:0/3295313545 conn(0x7f2f880cec10 msgr2=0x7f2f8815fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.130+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/3295313545 shutdown_connections 2023-11-24T06:17:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.131+0000 7f2f8e5a5640 1 -- 172.21.15.5:0/3295313545 wait complete. 2023-11-24T06:17:41.151 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T06:17:41.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.378+0000 7f4c9759e640 1 Processor -- start 2023-11-24T06:17:41.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.378+0000 7f4c9759e640 1 -- start start 2023-11-24T06:17:41.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.379+0000 7f4c9759e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4c9815d3b0 0x7f4c980ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:41.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.379+0000 7f4c9759e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4c980d0fa0 0x7f4c980d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:41.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.379+0000 7f4c9759e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c980c6bf0 con 0x7f4c980d1bc0 2023-11-24T06:17:41.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.379+0000 7f4c9759e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c980d1430 con 0x7f4c9815d3b0 2023-11-24T06:17:41.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.379+0000 7f4c9759e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c980d1fa0 con 0x7f4c980d0fa0 2023-11-24T06:17:41.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.380+0000 7f4c9659c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4c9815d3b0 0x7f4c980ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:41.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.380+0000 7f4c9659c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4c9815d3b0 0x7f4c980ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37312/0 (socket says 172.21.15.5:37312) 2023-11-24T06:17:41.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.380+0000 7f4c9659c640 1 -- 172.21.15.5:0/1114090193 learned_addr learned my addr 172.21.15.5:0/1114090193 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:41.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.380+0000 7f4c95d9b640 1 --2- 172.21.15.5:0/1114090193 >> v2:172.21.15.6:3300/0 conn(0x7f4c980d0fa0 0x7f4c980d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:41.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.380+0000 7f4c9659c640 1 -- 172.21.15.5:0/1114090193 >> v2:172.21.15.6:3300/0 conn(0x7f4c980d0fa0 msgr2=0x7f4c980d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:41.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.380+0000 7f4c9659c640 1 --2- 172.21.15.5:0/1114090193 >> v2:172.21.15.6:3300/0 conn(0x7f4c980d0fa0 0x7f4c980d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.380+0000 7f4c9659c640 1 -- 172.21.15.5:0/1114090193 >> v1:172.21.15.6:6789/0 conn(0x7f4c980d1bc0 legacy=0x7f4c980b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:41.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.380+0000 7f4c9659c640 1 -- 172.21.15.5:0/1114090193 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c980cedc0 con 0x7f4c9815d3b0 2023-11-24T06:17:41.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.381+0000 7f4c9659c640 1 --2- 172.21.15.5:0/1114090193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c9815d3b0 0x7f4c980ce6c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4c9808c4a0 tx=0x7f4c8802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c6474f9fcee831 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:41.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.382+0000 7f4c9559a640 1 -- 172.21.15.5:0/1114090193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8802c9f0 con 0x7f4c9815d3b0 2023-11-24T06:17:41.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.382+0000 7f4c9559a640 1 -- 172.21.15.5:0/1114090193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8802cb50 con 0x7f4c9815d3b0 2023-11-24T06:17:41.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.382+0000 7f4c9559a640 1 -- 172.21.15.5:0/1114090193 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c880343f0 con 0x7f4c9815d3b0 2023-11-24T06:17:41.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.382+0000 7f4c9759e640 1 -- 172.21.15.5:0/1114090193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c9815d3b0 msgr2=0x7f4c980ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:41.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.382+0000 7f4c9759e640 1 --2- 172.21.15.5:0/1114090193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c9815d3b0 0x7f4c980ce6c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4c9808c4a0 tx=0x7f4c8802be50 comp rx=0 tx=0).stop 2023-11-24T06:17:41.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.382+0000 7f4c9759e640 1 -- 172.21.15.5:0/1114090193 shutdown_connections 2023-11-24T06:17:41.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.382+0000 7f4c9759e640 1 --2- 172.21.15.5:0/1114090193 >> v2:172.21.15.6:3300/0 conn(0x7f4c980d0fa0 0x7f4c980d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.382+0000 7f4c9759e640 1 --2- 172.21.15.5:0/1114090193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c9815d3b0 0x7f4c980ce6c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.382+0000 7f4c9759e640 1 -- 172.21.15.5:0/1114090193 >> 172.21.15.5:0/1114090193 conn(0x7f4c980d30a0 msgr2=0x7f4c980d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:41.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.382+0000 7f4c9759e640 1 -- 172.21.15.5:0/1114090193 shutdown_connections 2023-11-24T06:17:41.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.383+0000 7f4c9759e640 1 -- 172.21.15.5:0/1114090193 wait complete. 2023-11-24T06:17:41.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.384+0000 7f4c9759e640 1 Processor -- start 2023-11-24T06:17:41.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.384+0000 7f4c9759e640 1 -- start start 2023-11-24T06:17:41.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.385+0000 7f4c9759e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c980d0fa0 0x7f4c980b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:41.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.385+0000 7f4c9759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c980d1bc0 0x7f4c9809ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:41.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.385+0000 7f4c9759e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c980b5c80 0x7f4c98153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:41.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.385+0000 7f4c9759e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980b6110 con 0x7f4c980d1bc0 2023-11-24T06:17:41.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.385+0000 7f4c9759e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980b63f0 con 0x7f4c980d0fa0 2023-11-24T06:17:41.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.385+0000 7f4c9759e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980b66d0 con 0x7f4c980b5c80 2023-11-24T06:17:41.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.386+0000 7f4c95d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c980d1bc0 0x7f4c9809ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:41.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.386+0000 7f4c95d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c980d1bc0 0x7f4c9809ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37326/0 (socket says 172.21.15.5:37326) 2023-11-24T06:17:41.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.386+0000 7f4c95d9b640 1 -- 172.21.15.5:0/1556670339 learned_addr learned my addr 172.21.15.5:0/1556670339 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:41.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.386+0000 7f4c9659c640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c980d0fa0 0x7f4c980b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:41.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.386+0000 7f4c96d9d640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c980b5c80 0x7f4c98153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:41.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.386+0000 7f4c95d9b640 1 -- 172.21.15.5:0/1556670339 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c980b5c80 msgr2=0x7f4c98153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:41.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.386+0000 7f4c95d9b640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c980b5c80 0x7f4c98153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.386+0000 7f4c95d9b640 1 -- 172.21.15.5:0/1556670339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c980d0fa0 msgr2=0x7f4c980b71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:41.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.386+0000 7f4c95d9b640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c980d0fa0 0x7f4c980b71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.386+0000 7f4c95d9b640 1 -- 172.21.15.5:0/1556670339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c88009b30 con 0x7f4c980d1bc0 2023-11-24T06:17:41.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.386+0000 7f4c96d9d640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c980b5c80 0x7f4c98153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:41.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.386+0000 7f4c95d9b640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c980d1bc0 0x7f4c9809ed30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000d5f0 tx=0x7f4c8000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:41.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.387+0000 7f4c8effd640 1 -- 172.21.15.5:0/1556670339 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c800125f0 con 0x7f4c980d1bc0 2023-11-24T06:17:41.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.387+0000 7f4c8effd640 1 -- 172.21.15.5:0/1556670339 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c80012750 con 0x7f4c980d1bc0 2023-11-24T06:17:41.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.387+0000 7f4c8effd640 1 -- 172.21.15.5:0/1556670339 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80012b00 con 0x7f4c980d1bc0 2023-11-24T06:17:41.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.387+0000 7f4c9759e640 1 -- 172.21.15.5:0/1556670339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c980b69b0 con 0x7f4c980d1bc0 2023-11-24T06:17:41.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.387+0000 7f4c9759e640 1 -- 172.21.15.5:0/1556670339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9815c880 con 0x7f4c980d1bc0 2023-11-24T06:17:41.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.388+0000 7f4c9759e640 1 -- 172.21.15.5:0/1556670339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9804e840 con 0x7f4c980d1bc0 2023-11-24T06:17:41.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.389+0000 7f4c8effd640 1 -- 172.21.15.5:0/1556670339 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f4c80012c60 con 0x7f4c980d1bc0 2023-11-24T06:17:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.390+0000 7f4c8effd640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4c7806aed0 0x7f4c7806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.390+0000 7f4c8effd640 1 -- 172.21.15.5:0/1556670339 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c80089c30 con 0x7f4c980d1bc0 2023-11-24T06:17:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.390+0000 7f4c9659c640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4c7806aed0 0x7f4c7806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.391+0000 7f4c9659c640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4c7806aed0 0x7f4c7806d390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4c88035000 tx=0x7f4c88030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.395+0000 7f4c8effd640 1 -- 172.21.15.5:0/1556670339 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c80054bf0 con 0x7f4c980d1bc0 2023-11-24T06:17:41.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.592+0000 7f4c9759e640 1 -- 172.21.15.5:0/1556670339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4c980d04c0 con 0x7f4c980d1bc0 2023-11-24T06:17:41.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.593+0000 7f4c8effd640 1 -- 172.21.15.5:0/1556670339 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4c80058210 con 0x7f4c980d1bc0 2023-11-24T06:17:41.596 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-11-24T06:17:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.596+0000 7f4c9759e640 1 -- 172.21.15.5:0/1556670339 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4c7806aed0 msgr2=0x7f4c7806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.596+0000 7f4c9759e640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4c7806aed0 0x7f4c7806d390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4c88035000 tx=0x7f4c88030000 comp rx=0 tx=0).stop 2023-11-24T06:17:41.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.596+0000 7f4c9759e640 1 -- 172.21.15.5:0/1556670339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c980d1bc0 msgr2=0x7f4c9809ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:41.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.596+0000 7f4c9759e640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c980d1bc0 0x7f4c9809ed30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000d5f0 tx=0x7f4c8000dab0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.596+0000 7f4c9759e640 1 -- 172.21.15.5:0/1556670339 shutdown_connections 2023-11-24T06:17:41.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.596+0000 7f4c9759e640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4c7806aed0 0x7f4c7806d390 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.596+0000 7f4c9759e640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4c980b5c80 0x7f4c98153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.596+0000 7f4c9759e640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c980d1bc0 0x7f4c9809ed30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.596+0000 7f4c9759e640 1 --2- 172.21.15.5:0/1556670339 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4c980d0fa0 0x7f4c980b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.596+0000 7f4c9759e640 1 -- 172.21.15.5:0/1556670339 >> 172.21.15.5:0/1556670339 conn(0x7f4c980d30a0 msgr2=0x7f4c980c7820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:41.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.596+0000 7f4c9759e640 1 -- 172.21.15.5:0/1556670339 shutdown_connections 2023-11-24T06:17:41.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.597+0000 7f4c9759e640 1 -- 172.21.15.5:0/1556670339 wait complete. 2023-11-24T06:17:41.618 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T06:17:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.846+0000 7f144359e640 1 Processor -- start 2023-11-24T06:17:41.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.847+0000 7f144359e640 1 -- start start 2023-11-24T06:17:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.848+0000 7f144359e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14440b3080 0x7f1444161ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.848+0000 7f144359e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f144406da40 0x7f144406c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.848+0000 7f144359e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144406c9a0 con 0x7f144415efa0 2023-11-24T06:17:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.848+0000 7f144359e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144406cc80 con 0x7f144406da40 2023-11-24T06:17:41.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.848+0000 7f144359e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f144406cf60 con 0x7f14440b3080 2023-11-24T06:17:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.848+0000 7f144259c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f144415efa0 0x7f14440b2970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35682/0 (socket says 172.21.15.5:35682) 2023-11-24T06:17:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.848+0000 7f144259c640 1 -- 172.21.15.5:0/2451704553 learned_addr learned my addr 172.21.15.5:0/2451704553 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.849+0000 7f1442d9d640 1 --2- 172.21.15.5:0/2451704553 >> v2:172.21.15.5:3300/0 conn(0x7f144406da40 0x7f144406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.849+0000 7f143bfff640 1 --2- 172.21.15.5:0/2451704553 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14440b3080 0x7f1444161ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.849+0000 7f1441d9b640 1 -- 172.21.15.5:0/2451704553 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997277251 0 0) 0x7f144406c9a0 con 0x7f144415efa0 2023-11-24T06:17:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.849+0000 7f1441d9b640 1 -- 172.21.15.5:0/2451704553 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1430003520 con 0x7f144415efa0 2023-11-24T06:17:41.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.849+0000 7f1442d9d640 1 -- 172.21.15.5:0/2451704553 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14440b3080 msgr2=0x7f1444161ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.849+0000 7f1442d9d640 1 --2- 172.21.15.5:0/2451704553 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14440b3080 0x7f1444161ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.849+0000 7f1442d9d640 1 -- 172.21.15.5:0/2451704553 >> v1:172.21.15.5:6789/0 conn(0x7f144415efa0 legacy=0x7f14440b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.849+0000 7f1442d9d640 1 -- 172.21.15.5:0/2451704553 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144406d240 con 0x7f144406da40 2023-11-24T06:17:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.850+0000 7f1442d9d640 1 --2- 172.21.15.5:0/2451704553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144406da40 0x7f144406c310 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f143c009d90 tx=0x7f143c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0c63a8d296a50d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:41.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.851+0000 7f1441d9b640 1 -- 172.21.15.5:0/2451704553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c02cc50 con 0x7f144406da40 2023-11-24T06:17:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.851+0000 7f1441d9b640 1 -- 172.21.15.5:0/2451704553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c02cdb0 con 0x7f144406da40 2023-11-24T06:17:41.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.851+0000 7f1441d9b640 1 -- 172.21.15.5:0/2451704553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143c034640 con 0x7f144406da40 2023-11-24T06:17:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.851+0000 7f144359e640 1 -- 172.21.15.5:0/2451704553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144406da40 msgr2=0x7f144406c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.851+0000 7f144359e640 1 --2- 172.21.15.5:0/2451704553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144406da40 0x7f144406c310 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f143c009d90 tx=0x7f143c02bea0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.851+0000 7f144359e640 1 -- 172.21.15.5:0/2451704553 shutdown_connections 2023-11-24T06:17:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.851+0000 7f144359e640 1 --2- 172.21.15.5:0/2451704553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144406da40 0x7f144406c310 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.851+0000 7f144359e640 1 --2- 172.21.15.5:0/2451704553 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14440b3080 0x7f1444161ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.851+0000 7f144359e640 1 -- 172.21.15.5:0/2451704553 >> 172.21.15.5:0/2451704553 conn(0x7f14440cec10 msgr2=0x7f14440cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.852+0000 7f144359e640 1 -- 172.21.15.5:0/2451704553 shutdown_connections 2023-11-24T06:17:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.852+0000 7f144359e640 1 -- 172.21.15.5:0/2451704553 wait complete. 2023-11-24T06:17:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.853+0000 7f144359e640 1 Processor -- start 2023-11-24T06:17:41.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.854+0000 7f144359e640 1 -- start start 2023-11-24T06:17:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.854+0000 7f144359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144406da40 0x7f14441f9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.854+0000 7f144359e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14440b3080 0x7f14440b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.854+0000 7f144359e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f144415efa0 0x7f14441fa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.854+0000 7f144359e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14440b0e10 con 0x7f144406da40 2023-11-24T06:17:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.854+0000 7f144359e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14441f9e40 con 0x7f144415efa0 2023-11-24T06:17:41.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.854+0000 7f144359e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14440b0190 con 0x7f14440b3080 2023-11-24T06:17:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.855+0000 7f144259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144406da40 0x7f14441f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.855+0000 7f144259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144406da40 0x7f14441f9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37352/0 (socket says 172.21.15.5:37352) 2023-11-24T06:17:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.855+0000 7f144259c640 1 -- 172.21.15.5:0/253661655 learned_addr learned my addr 172.21.15.5:0/253661655 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.855+0000 7f1442d9d640 1 --2- 172.21.15.5:0/253661655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f144415efa0 0x7f14441fa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.855+0000 7f143bfff640 1 --2- 172.21.15.5:0/253661655 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14440b3080 0x7f14440b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.855+0000 7f144259c640 1 -- 172.21.15.5:0/253661655 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14440b3080 msgr2=0x7f14440b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.855+0000 7f144259c640 1 --2- 172.21.15.5:0/253661655 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14440b3080 0x7f14440b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.855+0000 7f144259c640 1 -- 172.21.15.5:0/253661655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f144415efa0 msgr2=0x7f14441fa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.855+0000 7f144259c640 1 --2- 172.21.15.5:0/253661655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f144415efa0 0x7f14441fa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.855+0000 7f144259c640 1 -- 172.21.15.5:0/253661655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c009a70 con 0x7f144406da40 2023-11-24T06:17:41.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.856+0000 7f144259c640 1 --2- 172.21.15.5:0/253661655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144406da40 0x7f14441f9900 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f143400c390 tx=0x7f143400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.856+0000 7f143affd640 1 -- 172.21.15.5:0/253661655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434005ce0 con 0x7f144406da40 2023-11-24T06:17:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.856+0000 7f143affd640 1 -- 172.21.15.5:0/253661655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1434005e40 con 0x7f144406da40 2023-11-24T06:17:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.856+0000 7f143affd640 1 -- 172.21.15.5:0/253661655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14340072f0 con 0x7f144406da40 2023-11-24T06:17:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.856+0000 7f144359e640 1 -- 172.21.15.5:0/253661655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14440b0470 con 0x7f144406da40 2023-11-24T06:17:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.856+0000 7f144359e640 1 -- 172.21.15.5:0/253661655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14440b0a30 con 0x7f144406da40 2023-11-24T06:17:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.857+0000 7f144359e640 1 -- 172.21.15.5:0/253661655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144404e840 con 0x7f144406da40 2023-11-24T06:17:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.858+0000 7f143affd640 1 -- 172.21.15.5:0/253661655 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1434007450 con 0x7f144406da40 2023-11-24T06:17:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.859+0000 7f143affd640 1 --2- 172.21.15.5:0/253661655 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f142406aed0 0x7f142406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.859+0000 7f143affd640 1 -- 172.21.15.5:0/253661655 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1434088bd0 con 0x7f144406da40 2023-11-24T06:17:41.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.860+0000 7f143bfff640 1 --2- 172.21.15.5:0/253661655 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f142406aed0 0x7f142406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:41.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.860+0000 7f143bfff640 1 --2- 172.21.15.5:0/253661655 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f142406aed0 0x7f142406d390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1444163590 tx=0x7f142c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:41.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:41.864+0000 7f143affd640 1 -- 172.21.15.5:0/253661655 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1434053d20 con 0x7f144406da40 2023-11-24T06:17:42.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.056+0000 7f144359e640 1 -- 172.21.15.5:0/253661655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f144406b450 con 0x7f144406da40 2023-11-24T06:17:42.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.057+0000 7f143affd640 1 -- 172.21.15.5:0/253661655 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1434017020 con 0x7f144406da40 2023-11-24T06:17:42.060 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-11-24T06:17:42.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.060+0000 7f144359e640 1 -- 172.21.15.5:0/253661655 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f142406aed0 msgr2=0x7f142406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:42.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.060+0000 7f144359e640 1 --2- 172.21.15.5:0/253661655 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f142406aed0 0x7f142406d390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1444163590 tx=0x7f142c008040 comp rx=0 tx=0).stop 2023-11-24T06:17:42.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.060+0000 7f144359e640 1 -- 172.21.15.5:0/253661655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144406da40 msgr2=0x7f14441f9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:42.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.060+0000 7f144359e640 1 --2- 172.21.15.5:0/253661655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144406da40 0x7f14441f9900 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f143400c390 tx=0x7f143400c850 comp rx=0 tx=0).stop 2023-11-24T06:17:42.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.060+0000 7f144359e640 1 -- 172.21.15.5:0/253661655 shutdown_connections 2023-11-24T06:17:42.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.060+0000 7f144359e640 1 --2- 172.21.15.5:0/253661655 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f142406aed0 0x7f142406d390 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.060+0000 7f144359e640 1 --2- 172.21.15.5:0/253661655 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f14440b3080 0x7f14440b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.060+0000 7f144359e640 1 --2- 172.21.15.5:0/253661655 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f144415efa0 0x7f14441fa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.060+0000 7f144359e640 1 --2- 172.21.15.5:0/253661655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f144406da40 0x7f14441f9900 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.060+0000 7f144359e640 1 -- 172.21.15.5:0/253661655 >> 172.21.15.5:0/253661655 conn(0x7f14440cec10 msgr2=0x7f144415f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:42.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.060+0000 7f144359e640 1 -- 172.21.15.5:0/253661655 shutdown_connections 2023-11-24T06:17:42.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.060+0000 7f144359e640 1 -- 172.21.15.5:0/253661655 wait complete. 2023-11-24T06:17:42.082 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:17:42.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.295+0000 7f6191a9b640 1 Processor -- start 2023-11-24T06:17:42.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.295+0000 7f6191a9b640 1 -- start start 2023-11-24T06:17:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.297+0000 7f6191a9b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f618c06d520 0x7f618c06d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.297+0000 7f6191a9b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f618c0ad720 0x7f618c15ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.297+0000 7f6191a9b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f618c15c360 con 0x7f618c0abff0 2023-11-24T06:17:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.297+0000 7f6191a9b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f618c0b41a0 con 0x7f618c06d520 2023-11-24T06:17:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.297+0000 7f6191a9b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c06de40 con 0x7f618c0ad720 2023-11-24T06:17:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.298+0000 7f618b7fe640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f618c06d520 0x7f618c06d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.298+0000 7f618affd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f618c0ad720 0x7f618c15ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.298+0000 7f618b7fe640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f618c06d520 0x7f618c06d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:36654/0 (socket says 172.21.15.5:36654) 2023-11-24T06:17:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.298+0000 7f618affd640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f618c0ad720 0x7f618c15ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3301/0 says I am v2:172.21.15.5:60726/0 (socket says 172.21.15.5:60726) 2023-11-24T06:17:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.298+0000 7f618b7fe640 1 -- 172.21.15.5:0/1666782892 learned_addr learned my addr 172.21.15.5:0/1666782892 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.299+0000 7f618b7fe640 1 -- 172.21.15.5:0/1666782892 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f618c0ad720 msgr2=0x7f618c15ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.299+0000 7f618b7fe640 1 --2- 172.21.15.5:0/1666782892 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f618c0ad720 0x7f618c15ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.299+0000 7f618b7fe640 1 -- 172.21.15.5:0/1666782892 >> v1:172.21.15.6:6789/0 conn(0x7f618c0abff0 legacy=0x7f618c15f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.299+0000 7f618b7fe640 1 -- 172.21.15.5:0/1666782892 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c15ba30 con 0x7f618c06d520 2023-11-24T06:17:42.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.300+0000 7f618b7fe640 1 --2- 172.21.15.5:0/1666782892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618c06d520 0x7f618c06d900 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f617c009e50 tx=0x7f617c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5532e558ac1434b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:42.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.300+0000 7f618a7fc640 1 -- 172.21.15.5:0/1666782892 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c02cb00 con 0x7f618c06d520 2023-11-24T06:17:42.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.301+0000 7f618a7fc640 1 -- 172.21.15.5:0/1666782892 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c02cc60 con 0x7f618c06d520 2023-11-24T06:17:42.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.301+0000 7f618a7fc640 1 -- 172.21.15.5:0/1666782892 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c0343f0 con 0x7f618c06d520 2023-11-24T06:17:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.301+0000 7f6191a9b640 1 -- 172.21.15.5:0/1666782892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618c06d520 msgr2=0x7f618c06d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.301+0000 7f6191a9b640 1 --2- 172.21.15.5:0/1666782892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618c06d520 0x7f618c06d900 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f617c009e50 tx=0x7f617c02bf60 comp rx=0 tx=0).stop 2023-11-24T06:17:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.301+0000 7f6191a9b640 1 -- 172.21.15.5:0/1666782892 shutdown_connections 2023-11-24T06:17:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.301+0000 7f6191a9b640 1 --2- 172.21.15.5:0/1666782892 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f618c0ad720 0x7f618c15ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.301+0000 7f6191a9b640 1 --2- 172.21.15.5:0/1666782892 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618c06d520 0x7f618c06d900 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.301+0000 7f6191a9b640 1 -- 172.21.15.5:0/1666782892 >> 172.21.15.5:0/1666782892 conn(0x7f618c0b0d80 msgr2=0x7f618c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.302+0000 7f6191a9b640 1 -- 172.21.15.5:0/1666782892 shutdown_connections 2023-11-24T06:17:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.302+0000 7f6191a9b640 1 -- 172.21.15.5:0/1666782892 wait complete. 2023-11-24T06:17:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.303+0000 7f6191a9b640 1 Processor -- start 2023-11-24T06:17:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.303+0000 7f6191a9b640 1 -- start start 2023-11-24T06:17:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.304+0000 7f6191a9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f618c06d520 0x7f618c149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.304+0000 7f6191a9b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618c0abff0 0x7f618c0cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.304+0000 7f6191a9b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f618c0ad720 0x7f618c0cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.304+0000 7f6191a9b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c149c20 con 0x7f618c06d520 2023-11-24T06:17:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.304+0000 7f6191a9b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f618c0c9030 con 0x7f618c0abff0 2023-11-24T06:17:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.304+0000 7f6191a9b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f618c0c9310 con 0x7f618c0ad720 2023-11-24T06:17:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.305+0000 7f618b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f618c06d520 0x7f618c149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.305+0000 7f618b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f618c06d520 0x7f618c149260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37362/0 (socket says 172.21.15.5:37362) 2023-11-24T06:17:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.305+0000 7f618b7fe640 1 -- 172.21.15.5:0/2744408278 learned_addr learned my addr 172.21.15.5:0/2744408278 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.305+0000 7f618affd640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618c0abff0 0x7f618c0cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.305+0000 7f618bfff640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f618c0ad720 0x7f618c0cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.305+0000 7f618b7fe640 1 -- 172.21.15.5:0/2744408278 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f618c0ad720 msgr2=0x7f618c0cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.305+0000 7f618b7fe640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f618c0ad720 0x7f618c0cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.305+0000 7f618b7fe640 1 -- 172.21.15.5:0/2744408278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618c0abff0 msgr2=0x7f618c0cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.305+0000 7f618b7fe640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618c0abff0 0x7f618c0cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.305+0000 7f618b7fe640 1 -- 172.21.15.5:0/2744408278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617c009b30 con 0x7f618c06d520 2023-11-24T06:17:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.305+0000 7f618affd640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618c0abff0 0x7f618c0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.306+0000 7f618b7fe640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f618c06d520 0x7f618c149260 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f617c000f80 tx=0x7f617c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.306+0000 7f6183fff640 1 -- 172.21.15.5:0/2744408278 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c03e450 con 0x7f618c06d520 2023-11-24T06:17:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.306+0000 7f6183fff640 1 -- 172.21.15.5:0/2744408278 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f617c03e5f0 con 0x7f618c06d520 2023-11-24T06:17:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.306+0000 7f6183fff640 1 -- 172.21.15.5:0/2744408278 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f617c03e9a0 con 0x7f618c06d520 2023-11-24T06:17:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.306+0000 7f6191a9b640 1 -- 172.21.15.5:0/2744408278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f618c0c95f0 con 0x7f618c06d520 2023-11-24T06:17:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.306+0000 7f6191a9b640 1 -- 172.21.15.5:0/2744408278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f618c0c98d0 con 0x7f618c06d520 2023-11-24T06:17:42.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.307+0000 7f6191a9b640 1 -- 172.21.15.5:0/2744408278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f618c0b0620 con 0x7f618c06d520 2023-11-24T06:17:42.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.308+0000 7f6183fff640 1 -- 172.21.15.5:0/2744408278 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f617c03eb00 con 0x7f618c06d520 2023-11-24T06:17:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.309+0000 7f6183fff640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f616c06aed0 0x7f616c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.309+0000 7f6183fff640 1 -- 172.21.15.5:0/2744408278 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f617c0ac3f0 con 0x7f618c06d520 2023-11-24T06:17:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.309+0000 7f618affd640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f616c06aed0 0x7f616c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:42.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.310+0000 7f618affd640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f616c06aed0 0x7f616c06d390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f618c0b39b0 tx=0x7f6174008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.314+0000 7f6183fff640 1 -- 172.21.15.5:0/2744408278 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f617c0774d0 con 0x7f618c06d520 2023-11-24T06:17:42.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.546+0000 7f6191a9b640 1 -- 172.21.15.5:0/2744408278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f618c074770 con 0x7f618c06d520 2023-11-24T06:17:42.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.547+0000 7f6183fff640 1 -- 172.21.15.5:0/2744408278 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4901 (secure 0 0 0) 0x7f617c03edb0 con 0x7f618c06d520 2023-11-24T06:17:42.550 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:42.550 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6833/2072679798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2072679798},{"type":"v1","addr":"172.21.15.6:6833","nonce":2072679798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3855353235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3855353235},{"type":"v1","addr":"172.21.15.6:6835","nonce":3855353235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2394562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2394562290},{"type":"v1","addr":"172.21.15.5:6836","nonce":2394562290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:17:16.874899+0000","modified":"2023-11-24T06:17:20.326601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5217":{"gid":5217,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/3690265514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3690265514},{"type":"v1","addr":"172.21.15.5:6837","nonce":3690265514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:17:42.551 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 18 2023-11-24T06:17:42.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.550+0000 7f6191a9b640 1 -- 172.21.15.5:0/2744408278 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f616c06aed0 msgr2=0x7f616c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:42.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.550+0000 7f6191a9b640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f616c06aed0 0x7f616c06d390 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f618c0b39b0 tx=0x7f6174008040 comp rx=0 tx=0).stop 2023-11-24T06:17:42.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.550+0000 7f6191a9b640 1 -- 172.21.15.5:0/2744408278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f618c06d520 msgr2=0x7f618c149260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:42.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.550+0000 7f6191a9b640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f618c06d520 0x7f618c149260 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f617c000f80 tx=0x7f617c004a40 comp rx=0 tx=0).stop 2023-11-24T06:17:42.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.550+0000 7f6191a9b640 1 -- 172.21.15.5:0/2744408278 shutdown_connections 2023-11-24T06:17:42.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.550+0000 7f6191a9b640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f616c06aed0 0x7f616c06d390 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.550+0000 7f6191a9b640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f618c0ad720 0x7f618c0cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.550+0000 7f6191a9b640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f618c0abff0 0x7f618c0cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.550+0000 7f6191a9b640 1 --2- 172.21.15.5:0/2744408278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f618c06d520 0x7f618c149260 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.550+0000 7f6191a9b640 1 -- 172.21.15.5:0/2744408278 >> 172.21.15.5:0/2744408278 conn(0x7f618c0b0d80 msgr2=0x7f618c0aecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:42.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.550+0000 7f6191a9b640 1 -- 172.21.15.5:0/2744408278 shutdown_connections 2023-11-24T06:17:42.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.550+0000 7f6191a9b640 1 -- 172.21.15.5:0/2744408278 wait complete. 2023-11-24T06:17:42.572 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:17:42.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.788+0000 7f5d2759e640 1 Processor -- start 2023-11-24T06:17:42.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.788+0000 7f5d2759e640 1 -- start start 2023-11-24T06:17:42.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.789+0000 7f5d2759e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f5d2813dfd0 0x7f5d2815be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:42.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.789+0000 7f5d2759e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5d280cf500 0x7f5d2806c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.789+0000 7f5d2759e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d280ce7c0 con 0x7f5d280cf050 2023-11-24T06:17:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.789+0000 7f5d2759e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d28140900 con 0x7f5d280cf500 2023-11-24T06:17:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.789+0000 7f5d2759e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2815fa70 con 0x7f5d2813dfd0 2023-11-24T06:17:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.790+0000 7f5d26d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5d280cf500 0x7f5d2806c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.790+0000 7f5d26d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5d280cf500 0x7f5d2806c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37364/0 (socket says 172.21.15.5:37364) 2023-11-24T06:17:42.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.790+0000 7f5d26d9d640 1 -- 172.21.15.5:0/1771289690 learned_addr learned my addr 172.21.15.5:0/1771289690 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:42.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.790+0000 7f5d2659c640 1 --2- 172.21.15.5:0/1771289690 >> v2:172.21.15.6:3300/0 conn(0x7f5d2813dfd0 0x7f5d2815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:42.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.790+0000 7f5d26d9d640 1 -- 172.21.15.5:0/1771289690 >> v2:172.21.15.6:3300/0 conn(0x7f5d2813dfd0 msgr2=0x7f5d2815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:42.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.790+0000 7f5d26d9d640 1 --2- 172.21.15.5:0/1771289690 >> v2:172.21.15.6:3300/0 conn(0x7f5d2813dfd0 0x7f5d2815be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.790+0000 7f5d26d9d640 1 -- 172.21.15.5:0/1771289690 >> v1:172.21.15.6:6789/0 conn(0x7f5d280cf050 legacy=0x7f5d2806c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:42.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.790+0000 7f5d26d9d640 1 -- 172.21.15.5:0/1771289690 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2815ee60 con 0x7f5d280cf500 2023-11-24T06:17:42.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.791+0000 7f5d26d9d640 1 --2- 172.21.15.5:0/1771289690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d280cf500 0x7f5d2806c840 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5d280cfc10 tx=0x7f5d2002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74349494ff3cf183 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:42.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.792+0000 7f5d2559a640 1 -- 172.21.15.5:0/1771289690 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d20031870 con 0x7f5d280cf500 2023-11-24T06:17:42.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.792+0000 7f5d2559a640 1 -- 172.21.15.5:0/1771289690 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d200319d0 con 0x7f5d280cf500 2023-11-24T06:17:42.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.792+0000 7f5d2559a640 1 -- 172.21.15.5:0/1771289690 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d20031d60 con 0x7f5d280cf500 2023-11-24T06:17:42.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.792+0000 7f5d2759e640 1 -- 172.21.15.5:0/1771289690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d280cf500 msgr2=0x7f5d2806c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:42.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.792+0000 7f5d2759e640 1 --2- 172.21.15.5:0/1771289690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d280cf500 0x7f5d2806c840 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5d280cfc10 tx=0x7f5d2002b910 comp rx=0 tx=0).stop 2023-11-24T06:17:42.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.792+0000 7f5d2759e640 1 -- 172.21.15.5:0/1771289690 shutdown_connections 2023-11-24T06:17:42.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.792+0000 7f5d2759e640 1 --2- 172.21.15.5:0/1771289690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d280cf500 0x7f5d2806c840 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.792+0000 7f5d2759e640 1 --2- 172.21.15.5:0/1771289690 >> v2:172.21.15.6:3300/0 conn(0x7f5d2813dfd0 0x7f5d2815be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.792+0000 7f5d2759e640 1 -- 172.21.15.5:0/1771289690 >> 172.21.15.5:0/1771289690 conn(0x7f5d2806a860 msgr2=0x7f5d2806ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:42.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.792+0000 7f5d2759e640 1 -- 172.21.15.5:0/1771289690 shutdown_connections 2023-11-24T06:17:42.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.792+0000 7f5d2759e640 1 -- 172.21.15.5:0/1771289690 wait complete. 2023-11-24T06:17:42.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.794+0000 7f5d2759e640 1 Processor -- start 2023-11-24T06:17:42.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.794+0000 7f5d2759e640 1 -- start start 2023-11-24T06:17:42.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.795+0000 7f5d2759e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d280cf050 0x7f5d28070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.795+0000 7f5d2759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d280cf500 0x7f5d28153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.795+0000 7f5d2759e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d2813dfd0 0x7f5d28153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.795+0000 7f5d2759e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d28157f40 con 0x7f5d280cf500 2023-11-24T06:17:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.795+0000 7f5d2759e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d28071020 con 0x7f5d280cf050 2023-11-24T06:17:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.795+0000 7f5d2759e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d28157330 con 0x7f5d2813dfd0 2023-11-24T06:17:42.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.795+0000 7f5d25d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d280cf500 0x7f5d28153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.796+0000 7f5d25d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d280cf500 0x7f5d28153560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37368/0 (socket says 172.21.15.5:37368) 2023-11-24T06:17:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.796+0000 7f5d25d9b640 1 -- 172.21.15.5:0/2576241457 learned_addr learned my addr 172.21.15.5:0/2576241457 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.796+0000 7f5d2659c640 1 --2- 172.21.15.5:0/2576241457 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d280cf050 0x7f5d28070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.796+0000 7f5d25d9b640 1 -- 172.21.15.5:0/2576241457 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d2813dfd0 msgr2=0x7f5d28153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.796+0000 7f5d25d9b640 1 --2- 172.21.15.5:0/2576241457 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d2813dfd0 0x7f5d28153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.796+0000 7f5d25d9b640 1 -- 172.21.15.5:0/2576241457 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d280cf050 msgr2=0x7f5d28070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.796+0000 7f5d25d9b640 1 --2- 172.21.15.5:0/2576241457 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d280cf050 0x7f5d28070ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:42.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.796+0000 7f5d25d9b640 1 -- 172.21.15.5:0/2576241457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d200094d0 con 0x7f5d280cf500 2023-11-24T06:17:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.796+0000 7f5d2659c640 1 --2- 172.21.15.5:0/2576241457 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d280cf050 0x7f5d28070ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.796+0000 7f5d25d9b640 1 --2- 172.21.15.5:0/2576241457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d280cf500 0x7f5d28153560 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c410 tx=0x7f5d1c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.797+0000 7f5d16ffd640 1 -- 172.21.15.5:0/2576241457 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0035f0 con 0x7f5d280cf500 2023-11-24T06:17:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.797+0000 7f5d16ffd640 1 -- 172.21.15.5:0/2576241457 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c003750 con 0x7f5d280cf500 2023-11-24T06:17:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.797+0000 7f5d16ffd640 1 -- 172.21.15.5:0/2576241457 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c003b40 con 0x7f5d280cf500 2023-11-24T06:17:42.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.797+0000 7f5d2759e640 1 -- 172.21.15.5:0/2576241457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d28157610 con 0x7f5d280cf500 2023-11-24T06:17:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.797+0000 7f5d2759e640 1 -- 172.21.15.5:0/2576241457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d28157bd0 con 0x7f5d280cf500 2023-11-24T06:17:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.798+0000 7f5d2759e640 1 -- 172.21.15.5:0/2576241457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d281580b0 con 0x7f5d280cf500 2023-11-24T06:17:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.799+0000 7f5d16ffd640 1 -- 172.21.15.5:0/2576241457 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f5d1c003ca0 con 0x7f5d280cf500 2023-11-24T06:17:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.800+0000 7f5d16ffd640 1 --2- 172.21.15.5:0/2576241457 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5d0806aed0 0x7f5d0806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.800+0000 7f5d16ffd640 1 -- 172.21.15.5:0/2576241457 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d1c0888f0 con 0x7f5d280cf500 2023-11-24T06:17:42.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.800+0000 7f5d2659c640 1 --2- 172.21.15.5:0/2576241457 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5d0806aed0 0x7f5d0806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.801+0000 7f5d2659c640 1 --2- 172.21.15.5:0/2576241457 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5d0806aed0 0x7f5d0806d390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5d18007eb0 tx=0x7f5d18007400 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:42.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.804+0000 7f5d16ffd640 1 -- 172.21.15.5:0/2576241457 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d1c053940 con 0x7f5d280cf500 2023-11-24T06:17:43.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:42.999+0000 7f5d2759e640 1 -- 172.21.15.5:0/2576241457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d2804e840 con 0x7f5d280cf500 2023-11-24T06:17:43.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.001+0000 7f5d16ffd640 1 -- 172.21.15.5:0/2576241457 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16425 (secure 0 0 0) 0x7f5d1c056f60 con 0x7f5d280cf500 2023-11-24T06:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:43.005 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:17:18.380558+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:17:14.868917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:17:15.895189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:17:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.004+0000 7f5d2759e640 1 -- 172.21.15.5:0/2576241457 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5d0806aed0 msgr2=0x7f5d0806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.004+0000 7f5d2759e640 1 --2- 172.21.15.5:0/2576241457 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5d0806aed0 0x7f5d0806d390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5d18007eb0 tx=0x7f5d18007400 comp rx=0 tx=0).stop 2023-11-24T06:17:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.004+0000 7f5d2759e640 1 -- 172.21.15.5:0/2576241457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d280cf500 msgr2=0x7f5d28153560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.004+0000 7f5d2759e640 1 --2- 172.21.15.5:0/2576241457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d280cf500 0x7f5d28153560 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c410 tx=0x7f5d1c00c8d0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.005+0000 7f5d2759e640 1 -- 172.21.15.5:0/2576241457 shutdown_connections 2023-11-24T06:17:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.005+0000 7f5d2759e640 1 --2- 172.21.15.5:0/2576241457 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f5d0806aed0 0x7f5d0806d390 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.005+0000 7f5d2759e640 1 --2- 172.21.15.5:0/2576241457 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f5d2813dfd0 0x7f5d28153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.005+0000 7f5d2759e640 1 --2- 172.21.15.5:0/2576241457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d280cf500 0x7f5d28153560 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.005+0000 7f5d2759e640 1 --2- 172.21.15.5:0/2576241457 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f5d280cf050 0x7f5d28070ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.005+0000 7f5d2759e640 1 -- 172.21.15.5:0/2576241457 >> 172.21.15.5:0/2576241457 conn(0x7f5d2806a860 msgr2=0x7f5d2813c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.005+0000 7f5d2759e640 1 -- 172.21.15.5:0/2576241457 shutdown_connections 2023-11-24T06:17:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.005+0000 7f5d2759e640 1 -- 172.21.15.5:0/2576241457 wait complete. 2023-11-24T06:17:43.026 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:17:43.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:17:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.239+0000 7f7185cda640 1 Processor -- start 2023-11-24T06:17:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.239+0000 7f7185cda640 1 -- start start 2023-11-24T06:17:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.241+0000 7f7185cda640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f718015bfd0 0x7f71800cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.241+0000 7f7185cda640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f71800b4120 0x7f71800b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.241+0000 7f7185cda640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718015c460 con 0x7f7180158af0 2023-11-24T06:17:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.241+0000 7f7185cda640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71800b4af0 con 0x7f71800b4120 2023-11-24T06:17:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.241+0000 7f7185cda640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7180158ed0 con 0x7f718015bfd0 2023-11-24T06:17:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.241+0000 7f7184cd8640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7180158af0 0x7f718015c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43592/0 (socket says 172.21.15.5:43592) 2023-11-24T06:17:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.241+0000 7f7184cd8640 1 -- 172.21.15.5:0/1270531915 learned_addr learned my addr 172.21.15.5:0/1270531915 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.241+0000 7f71854d9640 1 --2- 172.21.15.5:0/1270531915 >> v2:172.21.15.5:3300/0 conn(0x7f71800b4120 0x7f71800b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.242+0000 7f717f7fe640 1 -- 172.21.15.5:0/1270531915 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704216037 0 0) 0x7f718015c460 con 0x7f7180158af0 2023-11-24T06:17:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.242+0000 7f71854d9640 1 -- 172.21.15.5:0/1270531915 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f718015bfd0 msgr2=0x7f71800cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.242+0000 7f71854d9640 1 --2- 172.21.15.5:0/1270531915 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f718015bfd0 0x7f71800cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.242+0000 7f71854d9640 1 -- 172.21.15.5:0/1270531915 >> v1:172.21.15.5:6789/0 conn(0x7f7180158af0 legacy=0x7f718015c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.242+0000 7f71854d9640 1 -- 172.21.15.5:0/1270531915 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71800cc7e0 con 0x7f71800b4120 2023-11-24T06:17:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.243+0000 7f71854d9640 1 --2- 172.21.15.5:0/1270531915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71800b4120 0x7f71800b45b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7178009d90 tx=0x7f717802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d062d470d2babc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.243+0000 7f717f7fe640 1 -- 172.21.15.5:0/1270531915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717802cc50 con 0x7f71800b4120 2023-11-24T06:17:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.243+0000 7f717f7fe640 1 -- 172.21.15.5:0/1270531915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717802cdb0 con 0x7f71800b4120 2023-11-24T06:17:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.243+0000 7f717f7fe640 1 -- 172.21.15.5:0/1270531915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7178034660 con 0x7f71800b4120 2023-11-24T06:17:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.244+0000 7f7185cda640 1 -- 172.21.15.5:0/1270531915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71800b4120 msgr2=0x7f71800b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.244+0000 7f7185cda640 1 --2- 172.21.15.5:0/1270531915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71800b4120 0x7f71800b45b0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7178009d90 tx=0x7f717802bea0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.244+0000 7f7185cda640 1 -- 172.21.15.5:0/1270531915 shutdown_connections 2023-11-24T06:17:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.244+0000 7f7185cda640 1 --2- 172.21.15.5:0/1270531915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71800b4120 0x7f71800b45b0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.244+0000 7f7185cda640 1 --2- 172.21.15.5:0/1270531915 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f718015bfd0 0x7f71800cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.244+0000 7f7185cda640 1 -- 172.21.15.5:0/1270531915 >> 172.21.15.5:0/1270531915 conn(0x7f71800b0b10 msgr2=0x7f71800b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.244+0000 7f7185cda640 1 -- 172.21.15.5:0/1270531915 shutdown_connections 2023-11-24T06:17:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.244+0000 7f7185cda640 1 -- 172.21.15.5:0/1270531915 wait complete. 2023-11-24T06:17:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.246+0000 7f7185cda640 1 Processor -- start 2023-11-24T06:17:43.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.246+0000 7f7185cda640 1 -- start start 2023-11-24T06:17:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f7185cda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71800b4120 0x7f71800c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f7185cda640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7180158af0 0x7f71800ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f7185cda640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f718015bfd0 0x7f71800a8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f7185cda640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718014d070 con 0x7f71800b4120 2023-11-24T06:17:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f7185cda640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71800adf80 con 0x7f718015bfd0 2023-11-24T06:17:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f7185cda640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718014cd90 con 0x7f7180158af0 2023-11-24T06:17:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f7184cd8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71800b4120 0x7f71800c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f7184cd8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71800b4120 0x7f71800c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50304/0 (socket says 172.21.15.5:50304) 2023-11-24T06:17:43.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f7184cd8640 1 -- 172.21.15.5:0/1510562161 learned_addr learned my addr 172.21.15.5:0/1510562161 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f71854d9640 1 --2- 172.21.15.5:0/1510562161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f718015bfd0 0x7f71800a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:43.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f7184cd8640 1 -- 172.21.15.5:0/1510562161 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7180158af0 msgr2=0x7f71800ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f7184cd8640 1 --2- 172.21.15.5:0/1510562161 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7180158af0 0x7f71800ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f7184cd8640 1 -- 172.21.15.5:0/1510562161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f718015bfd0 msgr2=0x7f71800a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f7184cd8640 1 --2- 172.21.15.5:0/1510562161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f718015bfd0 0x7f71800a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.247+0000 7f7184cd8640 1 -- 172.21.15.5:0/1510562161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7178009a70 con 0x7f71800b4120 2023-11-24T06:17:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.248+0000 7f7184cd8640 1 --2- 172.21.15.5:0/1510562161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71800b4120 0x7f71800c9c90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f717400c4d0 tx=0x7f717400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.248+0000 7f717d7fa640 1 -- 172.21.15.5:0/1510562161 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71740074e0 con 0x7f71800b4120 2023-11-24T06:17:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.248+0000 7f717d7fa640 1 -- 172.21.15.5:0/1510562161 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7174007660 con 0x7f71800b4120 2023-11-24T06:17:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.248+0000 7f717d7fa640 1 -- 172.21.15.5:0/1510562161 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174004a20 con 0x7f71800b4120 2023-11-24T06:17:43.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.248+0000 7f7185cda640 1 -- 172.21.15.5:0/1510562161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71800ae220 con 0x7f71800b4120 2023-11-24T06:17:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.248+0000 7f7185cda640 1 -- 172.21.15.5:0/1510562161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71800ae7e0 con 0x7f71800b4120 2023-11-24T06:17:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.249+0000 7f7185cda640 1 -- 172.21.15.5:0/1510562161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718004e840 con 0x7f71800b4120 2023-11-24T06:17:43.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.250+0000 7f717d7fa640 1 -- 172.21.15.5:0/1510562161 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f7174004b80 con 0x7f71800b4120 2023-11-24T06:17:43.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.251+0000 7f717d7fa640 1 --2- 172.21.15.5:0/1510562161 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f716406aed0 0x7f716406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:43.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.251+0000 7f717d7fa640 1 -- 172.21.15.5:0/1510562161 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7174088930 con 0x7f71800b4120 2023-11-24T06:17:43.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.251+0000 7f717ffff640 1 --2- 172.21.15.5:0/1510562161 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f716406aed0 0x7f716406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:43.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.252+0000 7f717ffff640 1 --2- 172.21.15.5:0/1510562161 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f716406aed0 0x7f716406d390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7180157070 tx=0x7f716c006bf0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.256+0000 7f717d7fa640 1 -- 172.21.15.5:0/1510562161 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f71740538f0 con 0x7f71800b4120 2023-11-24T06:17:43.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.490+0000 7f7185cda640 1 -- 172.21.15.5:0/1510562161 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f718015e240 con 0x7f71800b4120 2023-11-24T06:17:43.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.491+0000 7f717d7fa640 1 -- 172.21.15.5:0/1510562161 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4901 (secure 0 0 0) 0x7f7174018090 con 0x7f71800b4120 2023-11-24T06:17:43.494 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:43.494 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6833/2072679798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2072679798},{"type":"v1","addr":"172.21.15.6:6833","nonce":2072679798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3855353235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3855353235},{"type":"v1","addr":"172.21.15.6:6835","nonce":3855353235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2394562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2394562290},{"type":"v1","addr":"172.21.15.5:6836","nonce":2394562290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:17:16.874899+0000","modified":"2023-11-24T06:17:20.326601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5217":{"gid":5217,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/3690265514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3690265514},{"type":"v1","addr":"172.21.15.5:6837","nonce":3690265514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:17:43.495 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 18 2023-11-24T06:17:43.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.494+0000 7f7185cda640 1 -- 172.21.15.5:0/1510562161 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f716406aed0 msgr2=0x7f716406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.494+0000 7f7185cda640 1 --2- 172.21.15.5:0/1510562161 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f716406aed0 0x7f716406d390 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7180157070 tx=0x7f716c006bf0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.494+0000 7f7185cda640 1 -- 172.21.15.5:0/1510562161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71800b4120 msgr2=0x7f71800c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.494+0000 7f7185cda640 1 --2- 172.21.15.5:0/1510562161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71800b4120 0x7f71800c9c90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f717400c4d0 tx=0x7f717400c990 comp rx=0 tx=0).stop 2023-11-24T06:17:43.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.494+0000 7f7185cda640 1 -- 172.21.15.5:0/1510562161 shutdown_connections 2023-11-24T06:17:43.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.494+0000 7f7185cda640 1 --2- 172.21.15.5:0/1510562161 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f716406aed0 0x7f716406d390 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.494+0000 7f7185cda640 1 --2- 172.21.15.5:0/1510562161 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7180158af0 0x7f71800ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.494+0000 7f7185cda640 1 --2- 172.21.15.5:0/1510562161 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f718015bfd0 0x7f71800a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.494+0000 7f7185cda640 1 --2- 172.21.15.5:0/1510562161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71800b4120 0x7f71800c9c90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.494+0000 7f7185cda640 1 -- 172.21.15.5:0/1510562161 >> 172.21.15.5:0/1510562161 conn(0x7f71800b0b10 msgr2=0x7f718015c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:43.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.494+0000 7f7185cda640 1 -- 172.21.15.5:0/1510562161 shutdown_connections 2023-11-24T06:17:43.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.494+0000 7f7185cda640 1 -- 172.21.15.5:0/1510562161 wait complete. 2023-11-24T06:17:43.515 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:17:43.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.739+0000 7fb2e9cd3640 1 Processor -- start 2023-11-24T06:17:43.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.740+0000 7fb2e9cd3640 1 -- start start 2023-11-24T06:17:43.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.741+0000 7fb2e9cd3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2e40ad720 0x7fb2e40b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:43.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.741+0000 7fb2e9cd3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e415c360 con 0x7fb2e40abff0 2023-11-24T06:17:43.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.741+0000 7fb2e9cd3640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e415ba30 con 0x7fb2e406d520 2023-11-24T06:17:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.741+0000 7fb2e9cd3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40b41a0 con 0x7fb2e40ad720 2023-11-24T06:17:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.742+0000 7fb2e3fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb2e40abff0 0x7fb2e40b25c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43606/0 (socket says 172.21.15.5:43606) 2023-11-24T06:17:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.742+0000 7fb2e3fff640 1 -- 172.21.15.5:0/1256496857 learned_addr learned my addr 172.21.15.5:0/1256496857 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.742+0000 7fb2dbfff640 1 --2- 172.21.15.5:0/1256496857 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2e40ad720 0x7fb2e40b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.742+0000 7fb2e2ffd640 1 -- 172.21.15.5:0/1256496857 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322908742 0 0) 0x7fb2e415c360 con 0x7fb2e40abff0 2023-11-24T06:17:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.742+0000 7fb2e2ffd640 1 -- 172.21.15.5:0/1256496857 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2d0003520 con 0x7fb2e40abff0 2023-11-24T06:17:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2e2ffd640 1 -- 172.21.15.5:0/1256496857 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4256813274 0 0) 0x7fb2d0003520 con 0x7fb2e40abff0 2023-11-24T06:17:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2e2ffd640 1 -- 172.21.15.5:0/1256496857 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2e40ad720 msgr2=0x7fb2e40b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2e2ffd640 1 --2- 172.21.15.5:0/1256496857 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2e40ad720 0x7fb2e40b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2e2ffd640 1 -- 172.21.15.5:0/1256496857 >> v1:172.21.15.6:6789/0 conn(0x7fb2e406d520 legacy=0x7fb2e406d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2e2ffd640 1 -- 172.21.15.5:0/1256496857 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e415bd10 con 0x7fb2e40abff0 2023-11-24T06:17:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2e2ffd640 1 -- 172.21.15.5:0/1256496857 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fb2dc0019f0 con 0x7fb2e40abff0 2023-11-24T06:17:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2e2ffd640 1 -- 172.21.15.5:0/1256496857 >> v1:172.21.15.5:6789/0 conn(0x7fb2e40abff0 legacy=0x7fb2e40b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2e2ffd640 1 --2- 172.21.15.5:0/1256496857 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2e40ad720 0x7fb2d00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2e2ffd640 1 --2- 172.21.15.5:0/1256496857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d0003f70 0x7fb2d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2e2ffd640 1 --2- 172.21.15.5:0/1256496857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d0004d00 0x7fb2d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2e2ffd640 1 -- 172.21.15.5:0/1256496857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d000fea0 con 0x7fb2d0004d00 2023-11-24T06:17:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2e2ffd640 1 -- 172.21.15.5:0/1256496857 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d0010010 con 0x7fb2d0003f70 2023-11-24T06:17:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2e2ffd640 1 -- 172.21.15.5:0/1256496857 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d00101d0 con 0x7fb2e40ad720 2023-11-24T06:17:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2e3fff640 1 --2- 172.21.15.5:0/1256496857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d0004d00 0x7fb2d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.743+0000 7fb2dbfff640 1 --2- 172.21.15.5:0/1256496857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d0003f70 0x7fb2d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.744+0000 7fb2e37fe640 1 --2- 172.21.15.5:0/1256496857 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2e40ad720 0x7fb2d00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.744+0000 7fb2e3fff640 1 -- 172.21.15.5:0/1256496857 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2e40ad720 msgr2=0x7fb2d00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.744+0000 7fb2e3fff640 1 --2- 172.21.15.5:0/1256496857 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2e40ad720 0x7fb2d00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.744+0000 7fb2e3fff640 1 -- 172.21.15.5:0/1256496857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d0003f70 msgr2=0x7fb2d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.744+0000 7fb2e3fff640 1 --2- 172.21.15.5:0/1256496857 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d0003f70 0x7fb2d00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.744+0000 7fb2e3fff640 1 -- 172.21.15.5:0/1256496857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d0003520 con 0x7fb2d0004d00 2023-11-24T06:17:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.744+0000 7fb2e37fe640 1 --2- 172.21.15.5:0/1256496857 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2e40ad720 0x7fb2d00039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.745+0000 7fb2e3fff640 1 --2- 172.21.15.5:0/1256496857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d0004d00 0x7fb2d0005180 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40aca40 tx=0x7fb2d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bee25415002c61 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.745+0000 7fb2e2ffd640 1 -- 172.21.15.5:0/1256496857 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc034630 con 0x7fb2d0004d00 2023-11-24T06:17:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.745+0000 7fb2e2ffd640 1 -- 172.21.15.5:0/1256496857 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc02f070 con 0x7fb2d0004d00 2023-11-24T06:17:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.745+0000 7fb2e2ffd640 1 -- 172.21.15.5:0/1256496857 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0349d0 con 0x7fb2d0004d00 2023-11-24T06:17:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.746+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/1256496857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d0004d00 msgr2=0x7fb2d0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.746+0000 7fb2e9cd3640 1 --2- 172.21.15.5:0/1256496857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2d0004d00 0x7fb2d0005180 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40aca40 tx=0x7fb2d00057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.746+0000 7fb2e3fff640 1 -- 172.21.15.5:0/1256496857 reap_dead start 2023-11-24T06:17:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.746+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/1256496857 shutdown_connections 2023-11-24T06:17:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.746+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/1256496857 >> 172.21.15.5:0/1256496857 conn(0x7fb2e40b0d80 msgr2=0x7fb2e40b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.746+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/1256496857 shutdown_connections 2023-11-24T06:17:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.746+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/1256496857 wait complete. 2023-11-24T06:17:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.748+0000 7fb2e9cd3640 1 Processor -- start 2023-11-24T06:17:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.748+0000 7fb2e9cd3640 1 -- start start 2023-11-24T06:17:43.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.749+0000 7fb2e9cd3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d0004d00 0x7fb2e4098a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:43.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.749+0000 7fb2e9cd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2e40cbd40 0x7fb2e41457e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:43.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.749+0000 7fb2e9cd3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2e4071500 0x7fb2e4071980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:43.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.749+0000 7fb2e9cd3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e4071ec0 con 0x7fb2e40cbd40 2023-11-24T06:17:43.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.749+0000 7fb2e9cd3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e4142d40 con 0x7fb2d0004d00 2023-11-24T06:17:43.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.749+0000 7fb2e9cd3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e4145d20 con 0x7fb2e4071500 2023-11-24T06:17:43.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.749+0000 7fb2dbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2e40cbd40 0x7fb2e41457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:43.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.749+0000 7fb2dbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2e40cbd40 0x7fb2e41457e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50330/0 (socket says 172.21.15.5:50330) 2023-11-24T06:17:43.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.749+0000 7fb2dbfff640 1 -- 172.21.15.5:0/3699387900 learned_addr learned my addr 172.21.15.5:0/3699387900 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:43.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.750+0000 7fb2e37fe640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d0004d00 0x7fb2e4098a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:43.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.750+0000 7fb2e3fff640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2e4071500 0x7fb2e4071980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:43.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.750+0000 7fb2dbfff640 1 -- 172.21.15.5:0/3699387900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2e4071500 msgr2=0x7fb2e4071980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.750+0000 7fb2dbfff640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2e4071500 0x7fb2e4071980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.750+0000 7fb2dbfff640 1 -- 172.21.15.5:0/3699387900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d0004d00 msgr2=0x7fb2e4098a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.750+0000 7fb2dbfff640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d0004d00 0x7fb2e4098a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.750+0000 7fb2dbfff640 1 -- 172.21.15.5:0/3699387900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e40cc2e0 con 0x7fb2e40cbd40 2023-11-24T06:17:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.750+0000 7fb2e37fe640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d0004d00 0x7fb2e4098a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.750+0000 7fb2dbfff640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2e40cbd40 0x7fb2e41457e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc011560 tx=0x7fb2cc011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.750+0000 7fb2e0ff9640 1 -- 172.21.15.5:0/3699387900 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc004220 con 0x7fb2e40cbd40 2023-11-24T06:17:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.750+0000 7fb2e0ff9640 1 -- 172.21.15.5:0/3699387900 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2cc004380 con 0x7fb2e40cbd40 2023-11-24T06:17:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.750+0000 7fb2e0ff9640 1 -- 172.21.15.5:0/3699387900 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc015870 con 0x7fb2e40cbd40 2023-11-24T06:17:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.750+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/3699387900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e40cc5c0 con 0x7fb2e40cbd40 2023-11-24T06:17:43.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.750+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/3699387900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e40ccb80 con 0x7fb2e40cbd40 2023-11-24T06:17:43.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.752+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/3699387900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e4074770 con 0x7fb2e40cbd40 2023-11-24T06:17:43.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.753+0000 7fb2e0ff9640 1 -- 172.21.15.5:0/3699387900 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb2cc0159d0 con 0x7fb2e40cbd40 2023-11-24T06:17:43.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.753+0000 7fb2e0ff9640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb2c406aed0 0x7fb2c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:43.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.754+0000 7fb2e0ff9640 1 -- 172.21.15.5:0/3699387900 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2cc0898e0 con 0x7fb2e40cbd40 2023-11-24T06:17:43.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.754+0000 7fb2e37fe640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb2c406aed0 0x7fb2c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:43.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.754+0000 7fb2e37fe640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb2c406aed0 0x7fb2c406d390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4000c00 tx=0x7fb2d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:43.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.757+0000 7fb2e0ff9640 1 -- 172.21.15.5:0/3699387900 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb2cc054930 con 0x7fb2e40cbd40 2023-11-24T06:17:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.988+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/3699387900 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2e40ccf20 con 0x7fb2e40cbd40 2023-11-24T06:17:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.989+0000 7fb2e0ff9640 1 -- 172.21.15.5:0/3699387900 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4901 (secure 0 0 0) 0x7fb2cc057f50 con 0x7fb2e40cbd40 2023-11-24T06:17:43.992 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:43.992 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6833/2072679798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2072679798},{"type":"v1","addr":"172.21.15.6:6833","nonce":2072679798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3855353235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3855353235},{"type":"v1","addr":"172.21.15.6:6835","nonce":3855353235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2394562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2394562290},{"type":"v1","addr":"172.21.15.5:6836","nonce":2394562290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T06:17:16.874899+0000","modified":"2023-11-24T06:17:20.326601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5217},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5217":{"gid":5217,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.5:6837/3690265514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":3690265514},{"type":"v1","addr":"172.21.15.5:6837","nonce":3690265514}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T06:17:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 18 2023-11-24T06:17:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.991+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/3699387900 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb2c406aed0 msgr2=0x7fb2c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.991+0000 7fb2e9cd3640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb2c406aed0 0x7fb2c406d390 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4000c00 tx=0x7fb2d4009040 comp rx=0 tx=0).stop 2023-11-24T06:17:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.992+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/3699387900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2e40cbd40 msgr2=0x7fb2e41457e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.992+0000 7fb2e9cd3640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2e40cbd40 0x7fb2e41457e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc011560 tx=0x7fb2cc011a20 comp rx=0 tx=0).stop 2023-11-24T06:17:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.992+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/3699387900 shutdown_connections 2023-11-24T06:17:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.992+0000 7fb2e9cd3640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb2c406aed0 0x7fb2c406d390 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.992+0000 7fb2e9cd3640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb2e4071500 0x7fb2e4071980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.992+0000 7fb2e9cd3640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2e40cbd40 0x7fb2e41457e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.992+0000 7fb2e9cd3640 1 --2- 172.21.15.5:0/3699387900 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb2d0004d00 0x7fb2e4098a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.992+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/3699387900 >> 172.21.15.5:0/3699387900 conn(0x7fb2e40b0d80 msgr2=0x7fb2e40afe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.992+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/3699387900 shutdown_connections 2023-11-24T06:17:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:43.992+0000 7fb2e9cd3640 1 -- 172.21.15.5:0/3699387900 wait complete. 2023-11-24T06:17:44.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:17:44.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.235+0000 7fe53c6d3640 1 Processor -- start 2023-11-24T06:17:44.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.235+0000 7fe53c6d3640 1 -- start start 2023-11-24T06:17:44.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.237+0000 7fe53c6d3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe534161930 0x7fe5340ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:44.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.237+0000 7fe53c6d3640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe53406d520 0x7fe53406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:44.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.237+0000 7fe53c6d3640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5340ad510 con 0x7fe5340719c0 2023-11-24T06:17:44.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.237+0000 7fe53c6d3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5340acc50 con 0x7fe534161930 2023-11-24T06:17:44.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.237+0000 7fe53c6d3640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe53406de70 con 0x7fe53406d520 2023-11-24T06:17:44.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.237+0000 7fe53a46f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe534161930 0x7fe5340ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:44.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.238+0000 7fe53a46f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe534161930 0x7fe5340ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50348/0 (socket says 172.21.15.5:50348) 2023-11-24T06:17:44.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.238+0000 7fe53a46f640 1 -- 172.21.15.5:0/4250305900 learned_addr learned my addr 172.21.15.5:0/4250305900 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:44.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.238+0000 7fe53ac70640 1 --2- 172.21.15.5:0/4250305900 >> v2:172.21.15.6:3300/0 conn(0x7fe53406d520 0x7fe53406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:44.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.238+0000 7fe53a46f640 1 -- 172.21.15.5:0/4250305900 >> v2:172.21.15.6:3300/0 conn(0x7fe53406d520 msgr2=0x7fe53406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.238+0000 7fe53a46f640 1 --2- 172.21.15.5:0/4250305900 >> v2:172.21.15.6:3300/0 conn(0x7fe53406d520 0x7fe53406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.238+0000 7fe53a46f640 1 -- 172.21.15.5:0/4250305900 >> v1:172.21.15.6:6789/0 conn(0x7fe5340719c0 legacy=0x7fe5340ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.238+0000 7fe53a46f640 1 -- 172.21.15.5:0/4250305900 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe534071e60 con 0x7fe534161930 2023-11-24T06:17:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.240+0000 7fe53a46f640 1 --2- 172.21.15.5:0/4250305900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe534161930 0x7fe5340ae6a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe534070440 tx=0x7fe53002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd64e23f5f0c9ad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.240+0000 7fe53946d640 1 -- 172.21.15.5:0/4250305900 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530031870 con 0x7fe534161930 2023-11-24T06:17:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.240+0000 7fe53946d640 1 -- 172.21.15.5:0/4250305900 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5300319d0 con 0x7fe534161930 2023-11-24T06:17:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.240+0000 7fe53946d640 1 -- 172.21.15.5:0/4250305900 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530031d60 con 0x7fe534161930 2023-11-24T06:17:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.241+0000 7fe53c6d3640 1 -- 172.21.15.5:0/4250305900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe534161930 msgr2=0x7fe5340ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.241+0000 7fe53c6d3640 1 --2- 172.21.15.5:0/4250305900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe534161930 0x7fe5340ae6a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fe534070440 tx=0x7fe53002b910 comp rx=0 tx=0).stop 2023-11-24T06:17:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.241+0000 7fe53c6d3640 1 -- 172.21.15.5:0/4250305900 shutdown_connections 2023-11-24T06:17:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.241+0000 7fe53c6d3640 1 --2- 172.21.15.5:0/4250305900 >> v2:172.21.15.6:3300/0 conn(0x7fe53406d520 0x7fe53406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.241+0000 7fe53c6d3640 1 --2- 172.21.15.5:0/4250305900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe534161930 0x7fe5340ae6a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.241+0000 7fe53c6d3640 1 -- 172.21.15.5:0/4250305900 >> 172.21.15.5:0/4250305900 conn(0x7fe5340b0d80 msgr2=0x7fe5340b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:44.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.241+0000 7fe53c6d3640 1 -- 172.21.15.5:0/4250305900 shutdown_connections 2023-11-24T06:17:44.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.241+0000 7fe53c6d3640 1 -- 172.21.15.5:0/4250305900 wait complete. 2023-11-24T06:17:44.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.243+0000 7fe53c6d3640 1 Processor -- start 2023-11-24T06:17:44.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.243+0000 7fe53c6d3640 1 -- start start 2023-11-24T06:17:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.244+0000 7fe53c6d3640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe53406d520 0x7fe53415b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.244+0000 7fe53c6d3640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5340719c0 0x7fe53415d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.244+0000 7fe53c6d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe534161930 0x7fe53415ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.244+0000 7fe53c6d3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53415ebd0 con 0x7fe534161930 2023-11-24T06:17:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.244+0000 7fe53c6d3640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53415dfc0 con 0x7fe5340719c0 2023-11-24T06:17:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.244+0000 7fe53c6d3640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53415e2a0 con 0x7fe53406d520 2023-11-24T06:17:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.244+0000 7fe53ac70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe534161930 0x7fe53415ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.244+0000 7fe53ac70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe534161930 0x7fe53415ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50350/0 (socket says 172.21.15.5:50350) 2023-11-24T06:17:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.244+0000 7fe53ac70640 1 -- 172.21.15.5:0/1423504298 learned_addr learned my addr 172.21.15.5:0/1423504298 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.245+0000 7fe53ac70640 1 -- 172.21.15.5:0/1423504298 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe53406d520 msgr2=0x7fe53415b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.245+0000 7fe53ac70640 1 --2- 172.21.15.5:0/1423504298 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe53406d520 0x7fe53415b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.245+0000 7fe53ac70640 1 -- 172.21.15.5:0/1423504298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5340719c0 msgr2=0x7fe53415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.245+0000 7fe539c6e640 1 --2- 172.21.15.5:0/1423504298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5340719c0 0x7fe53415d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.245+0000 7fe53ac70640 1 --2- 172.21.15.5:0/1423504298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5340719c0 0x7fe53415d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.245+0000 7fe53ac70640 1 -- 172.21.15.5:0/1423504298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5300094d0 con 0x7fe534161930 2023-11-24T06:17:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.245+0000 7fe53ac70640 1 --2- 172.21.15.5:0/1423504298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe534161930 0x7fe53415ef00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe52c007e40 tx=0x7fe52c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.245+0000 7fe52affd640 1 -- 172.21.15.5:0/1423504298 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c014620 con 0x7fe534161930 2023-11-24T06:17:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.245+0000 7fe52affd640 1 -- 172.21.15.5:0/1423504298 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52c00ee90 con 0x7fe534161930 2023-11-24T06:17:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.245+0000 7fe52affd640 1 -- 172.21.15.5:0/1423504298 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0149e0 con 0x7fe534161930 2023-11-24T06:17:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.245+0000 7fe53c6d3640 1 -- 172.21.15.5:0/1423504298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53415e580 con 0x7fe534161930 2023-11-24T06:17:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.245+0000 7fe53c6d3640 1 -- 172.21.15.5:0/1423504298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53415b930 con 0x7fe534161930 2023-11-24T06:17:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.247+0000 7fe53c6d3640 1 -- 172.21.15.5:0/1423504298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe534074770 con 0x7fe534161930 2023-11-24T06:17:44.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.248+0000 7fe52affd640 1 -- 172.21.15.5:0/1423504298 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe52c014b40 con 0x7fe534161930 2023-11-24T06:17:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.249+0000 7fe52affd640 1 --2- 172.21.15.5:0/1423504298 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe51806af00 0x7fe51806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.249+0000 7fe52affd640 1 -- 172.21.15.5:0/1423504298 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe52c08a060 con 0x7fe534161930 2023-11-24T06:17:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.249+0000 7fe53a46f640 1 --2- 172.21.15.5:0/1423504298 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe51806af00 0x7fe51806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.250+0000 7fe53a46f640 1 --2- 172.21.15.5:0/1423504298 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe51806af00 0x7fe51806d3c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe530036000 tx=0x7fe530030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:44.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.253+0000 7fe52affd640 1 -- 172.21.15.5:0/1423504298 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe52c0bce90 con 0x7fe534161930 2023-11-24T06:17:44.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.449+0000 7fe53c6d3640 1 -- 172.21.15.5:0/1423504298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe534162040 con 0x7fe534161930 2023-11-24T06:17:44.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.452+0000 7fe52affd640 1 -- 172.21.15.5:0/1423504298 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16425 (secure 0 0 0) 0x7fe534162040 con 0x7fe534161930 2023-11-24T06:17:44.455 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:44.455 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:17:18.380558+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T06:17:14.868917+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T06:17:15.895189+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:17:44.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.454+0000 7fe53c6d3640 1 -- 172.21.15.5:0/1423504298 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe51806af00 msgr2=0x7fe51806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:44.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.454+0000 7fe53c6d3640 1 --2- 172.21.15.5:0/1423504298 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe51806af00 0x7fe51806d3c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe530036000 tx=0x7fe530030000 comp rx=0 tx=0).stop 2023-11-24T06:17:44.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.454+0000 7fe53c6d3640 1 -- 172.21.15.5:0/1423504298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe534161930 msgr2=0x7fe53415ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:44.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.454+0000 7fe53c6d3640 1 --2- 172.21.15.5:0/1423504298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe534161930 0x7fe53415ef00 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fe52c007e40 tx=0x7fe52c00e4b0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.455+0000 7fe53c6d3640 1 -- 172.21.15.5:0/1423504298 shutdown_connections 2023-11-24T06:17:44.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.455+0000 7fe53c6d3640 1 --2- 172.21.15.5:0/1423504298 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe51806af00 0x7fe51806d3c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.455+0000 7fe53c6d3640 1 --2- 172.21.15.5:0/1423504298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe534161930 0x7fe53415ef00 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.455+0000 7fe53c6d3640 1 --2- 172.21.15.5:0/1423504298 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe5340719c0 0x7fe53415d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.455+0000 7fe53c6d3640 1 --2- 172.21.15.5:0/1423504298 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe53406d520 0x7fe53415b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.455+0000 7fe53c6d3640 1 -- 172.21.15.5:0/1423504298 >> 172.21.15.5:0/1423504298 conn(0x7fe5340b0d80 msgr2=0x7fe5340ae140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:44.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.455+0000 7fe53c6d3640 1 -- 172.21.15.5:0/1423504298 shutdown_connections 2023-11-24T06:17:44.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.455+0000 7fe53c6d3640 1 -- 172.21.15.5:0/1423504298 wait complete. 2023-11-24T06:17:44.477 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T06:17:44.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.702+0000 7f912a167640 1 Processor -- start 2023-11-24T06:17:44.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.703+0000 7f912a167640 1 -- start start 2023-11-24T06:17:44.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.704+0000 7f912a167640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f91240d1d80 0x7f91240cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:44.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.704+0000 7f912a167640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91240d04b0 con 0x7f912415d5d0 2023-11-24T06:17:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.704+0000 7f912a167640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91240cdb80 con 0x7f91240cf710 2023-11-24T06:17:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.704+0000 7f912a167640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91240ce4e0 con 0x7f91240d1d80 2023-11-24T06:17:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.705+0000 7f9129165640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f912415d5d0 0x7f91240ce900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43614/0 (socket says 172.21.15.5:43614) 2023-11-24T06:17:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.705+0000 7f9129165640 1 -- 172.21.15.5:0/571458494 learned_addr learned my addr 172.21.15.5:0/571458494 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:44.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.705+0000 7f9129966640 1 --2- 172.21.15.5:0/571458494 >> v2:172.21.15.5:3300/0 conn(0x7f91240d1d80 0x7f91240cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:44.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.705+0000 7f9117fff640 1 -- 172.21.15.5:0/571458494 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220267375 0 0) 0x7f91240d04b0 con 0x7f912415d5d0 2023-11-24T06:17:44.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.705+0000 7f9117fff640 1 -- 172.21.15.5:0/571458494 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9110003520 con 0x7f912415d5d0 2023-11-24T06:17:44.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.705+0000 7f9129966640 1 -- 172.21.15.5:0/571458494 >> v1:172.21.15.6:6789/0 conn(0x7f91240cf710 legacy=0x7f91240d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:44.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.705+0000 7f9129966640 1 -- 172.21.15.5:0/571458494 >> v1:172.21.15.5:6789/0 conn(0x7f912415d5d0 legacy=0x7f91240ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:44.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.705+0000 7f9129966640 1 -- 172.21.15.5:0/571458494 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91240d01d0 con 0x7f91240d1d80 2023-11-24T06:17:44.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.706+0000 7f9129966640 1 --2- 172.21.15.5:0/571458494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91240d1d80 0x7f91240cfb70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f911c009d90 tx=0x7f911c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46d8025903df7c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:44.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.707+0000 7f9117fff640 1 -- 172.21.15.5:0/571458494 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c02cbd0 con 0x7f91240d1d80 2023-11-24T06:17:44.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.707+0000 7f9117fff640 1 -- 172.21.15.5:0/571458494 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c02cd30 con 0x7f91240d1d80 2023-11-24T06:17:44.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.707+0000 7f9117fff640 1 -- 172.21.15.5:0/571458494 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c02cea0 con 0x7f91240d1d80 2023-11-24T06:17:44.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.707+0000 7f912a167640 1 -- 172.21.15.5:0/571458494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91240d1d80 msgr2=0x7f91240cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:44.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.707+0000 7f912a167640 1 --2- 172.21.15.5:0/571458494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91240d1d80 0x7f91240cfb70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f911c009d90 tx=0x7f911c02be20 comp rx=0 tx=0).stop 2023-11-24T06:17:44.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.707+0000 7f912a167640 1 -- 172.21.15.5:0/571458494 shutdown_connections 2023-11-24T06:17:44.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.707+0000 7f912a167640 1 --2- 172.21.15.5:0/571458494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91240d1d80 0x7f91240cfb70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.707+0000 7f912a167640 1 -- 172.21.15.5:0/571458494 >> 172.21.15.5:0/571458494 conn(0x7f912409c3d0 msgr2=0x7f91240d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:44.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.707+0000 7f912a167640 1 -- 172.21.15.5:0/571458494 shutdown_connections 2023-11-24T06:17:44.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.708+0000 7f912a167640 1 -- 172.21.15.5:0/571458494 wait complete. 2023-11-24T06:17:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.709+0000 7f912a167640 1 Processor -- start 2023-11-24T06:17:44.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.709+0000 7f912a167640 1 -- start start 2023-11-24T06:17:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.710+0000 7f912a167640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91240cf710 0x7f91241571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.710+0000 7f912a167640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91240d1d80 0x7f91240cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.710+0000 7f912a167640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f912415d5d0 0x7f912415e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:44.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.710+0000 7f912a167640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91240cf010 con 0x7f91240cf710 2023-11-24T06:17:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.710+0000 7f912a167640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9124157f30 con 0x7f91240d1d80 2023-11-24T06:17:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.710+0000 7f912a167640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91240cc9a0 con 0x7f912415d5d0 2023-11-24T06:17:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.711+0000 7f9129165640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91240cf710 0x7f91241571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.711+0000 7f9129165640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91240cf710 0x7f91241571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50360/0 (socket says 172.21.15.5:50360) 2023-11-24T06:17:44.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.711+0000 7f9129165640 1 -- 172.21.15.5:0/1507387183 learned_addr learned my addr 172.21.15.5:0/1507387183 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.711+0000 7f9129966640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f912415d5d0 0x7f912415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.711+0000 7f9128964640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91240d1d80 0x7f91240cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.711+0000 7f9129165640 1 -- 172.21.15.5:0/1507387183 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f912415d5d0 msgr2=0x7f912415e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.711+0000 7f9129165640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f912415d5d0 0x7f912415e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.711+0000 7f9129165640 1 -- 172.21.15.5:0/1507387183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91240d1d80 msgr2=0x7f91240cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.711+0000 7f9129165640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91240d1d80 0x7f91240cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.711+0000 7f9129165640 1 -- 172.21.15.5:0/1507387183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c009a70 con 0x7f91240cf710 2023-11-24T06:17:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.711+0000 7f9129966640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f912415d5d0 0x7f912415e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.711+0000 7f9129165640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91240cf710 0x7f91241571b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f911800c410 tx=0x7f911800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.712+0000 7f9115ffb640 1 -- 172.21.15.5:0/1507387183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9118005ce0 con 0x7f91240cf710 2023-11-24T06:17:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.712+0000 7f9115ffb640 1 -- 172.21.15.5:0/1507387183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9118005e40 con 0x7f91240cf710 2023-11-24T06:17:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.712+0000 7f912a167640 1 -- 172.21.15.5:0/1507387183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91240ccc80 con 0x7f91240cf710 2023-11-24T06:17:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.712+0000 7f9115ffb640 1 -- 172.21.15.5:0/1507387183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91180072f0 con 0x7f91240cf710 2023-11-24T06:17:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.712+0000 7f912a167640 1 -- 172.21.15.5:0/1507387183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912415dd30 con 0x7f91240cf710 2023-11-24T06:17:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.713+0000 7f912a167640 1 -- 172.21.15.5:0/1507387183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91240d2a40 con 0x7f91240cf710 2023-11-24T06:17:44.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.714+0000 7f9115ffb640 1 -- 172.21.15.5:0/1507387183 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f9118007510 con 0x7f91240cf710 2023-11-24T06:17:44.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.715+0000 7f9115ffb640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f910806aed0 0x7f910806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:44.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.715+0000 7f9115ffb640 1 -- 172.21.15.5:0/1507387183 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9118088dc0 con 0x7f91240cf710 2023-11-24T06:17:44.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.715+0000 7f9128964640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f910806aed0 0x7f910806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:44.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.716+0000 7f9128964640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f910806aed0 0x7f910806d390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f91240cf5a0 tx=0x7f9120009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.719+0000 7f9115ffb640 1 -- 172.21.15.5:0/1507387183 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9118053e40 con 0x7f91240cf710 2023-11-24T06:17:44.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.947+0000 7f912a167640 1 -- 172.21.15.5:0/1507387183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f91240b59f0 con 0x7f91240cf710 2023-11-24T06:17:44.950 INFO:tasks.ceph.mon.a.smithi005.stderr:2023-11-24T06:17:44.948+0000 7f463fba2640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T06:17:44.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.951+0000 7f9115ffb640 1 -- 172.21.15.5:0/1507387183 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9118017090 con 0x7f91240cf710 2023-11-24T06:17:44.955 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T06:17:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.954+0000 7f912a167640 1 -- 172.21.15.5:0/1507387183 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f910806aed0 msgr2=0x7f910806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.954+0000 7f912a167640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f910806aed0 0x7f910806d390 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f91240cf5a0 tx=0x7f9120009040 comp rx=0 tx=0).stop 2023-11-24T06:17:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.954+0000 7f912a167640 1 -- 172.21.15.5:0/1507387183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91240cf710 msgr2=0x7f91241571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:44.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.954+0000 7f912a167640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91240cf710 0x7f91241571b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f911800c410 tx=0x7f911800c8d0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.955+0000 7f912a167640 1 -- 172.21.15.5:0/1507387183 shutdown_connections 2023-11-24T06:17:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.955+0000 7f912a167640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f910806aed0 0x7f910806d390 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.955+0000 7f912a167640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f912415d5d0 0x7f912415e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.955+0000 7f912a167640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f91240d1d80 0x7f91240cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.955+0000 7f912a167640 1 --2- 172.21.15.5:0/1507387183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91240cf710 0x7f91241571b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.955+0000 7f912a167640 1 -- 172.21.15.5:0/1507387183 >> 172.21.15.5:0/1507387183 conn(0x7f912409c3d0 msgr2=0x7f91240ce900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:44.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.955+0000 7f912a167640 1 -- 172.21.15.5:0/1507387183 shutdown_connections 2023-11-24T06:17:44.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:44.956+0000 7f912a167640 1 -- 172.21.15.5:0/1507387183 wait complete. 2023-11-24T06:17:44.981 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T06:17:45.046 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2023-11-24T06:17:45.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.170+0000 7f7ec308c640 1 Processor -- start 2023-11-24T06:17:45.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.170+0000 7f7ec308c640 1 -- start start 2023-11-24T06:17:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.171+0000 7f7ec308c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7ebc06d9e0 0x7f7ebc165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.171+0000 7f7ec308c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ebc15f930 con 0x7f7ebc144770 2023-11-24T06:17:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.171+0000 7f7ec308c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ebc0b3b10 con 0x7f7ebc15f270 2023-11-24T06:17:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.171+0000 7f7ec308c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ebc0b43b0 con 0x7f7ebc06d9e0 2023-11-24T06:17:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.172+0000 7f7ec288b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7ebc06d9e0 0x7f7ebc165ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.172+0000 7f7ec208a640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f7ebc144770 0x7f7ebc15eb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43626/0 (socket says 172.21.15.5:43626) 2023-11-24T06:17:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.172+0000 7f7ec208a640 1 -- 172.21.15.5:0/244544282 learned_addr learned my addr 172.21.15.5:0/244544282 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.172+0000 7f7ec1088640 1 -- 172.21.15.5:0/244544282 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517542908 0 0) 0x7f7ebc15f930 con 0x7f7ebc144770 2023-11-24T06:17:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.172+0000 7f7ec1088640 1 -- 172.21.15.5:0/244544282 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ea8003520 con 0x7f7ebc144770 2023-11-24T06:17:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.172+0000 7f7ec288b640 1 -- 172.21.15.5:0/244544282 >> v1:172.21.15.6:6789/0 conn(0x7f7ebc15f270 legacy=0x7f7ebc0cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.172+0000 7f7ec288b640 1 -- 172.21.15.5:0/244544282 >> v1:172.21.15.5:6789/0 conn(0x7f7ebc144770 legacy=0x7f7ebc15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.172+0000 7f7ec288b640 1 -- 172.21.15.5:0/244544282 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc0b3df0 con 0x7f7ebc06d9e0 2023-11-24T06:17:45.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.173+0000 7f7ec288b640 1 --2- 172.21.15.5:0/244544282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc06d9e0 0x7f7ebc165ef0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009d90 tx=0x7f7eb402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1c4dc5d9fc24fb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:45.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.173+0000 7f7ec1088640 1 -- 172.21.15.5:0/244544282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb402cc60 con 0x7f7ebc06d9e0 2023-11-24T06:17:45.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.174+0000 7f7ec1088640 1 -- 172.21.15.5:0/244544282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb402cdc0 con 0x7f7ebc06d9e0 2023-11-24T06:17:45.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.174+0000 7f7ec1088640 1 -- 172.21.15.5:0/244544282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4034660 con 0x7f7ebc06d9e0 2023-11-24T06:17:45.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.174+0000 7f7ec308c640 1 -- 172.21.15.5:0/244544282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc06d9e0 msgr2=0x7f7ebc165ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:45.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.174+0000 7f7ec308c640 1 --2- 172.21.15.5:0/244544282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc06d9e0 0x7f7ebc165ef0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009d90 tx=0x7f7eb402beb0 comp rx=0 tx=0).stop 2023-11-24T06:17:45.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.174+0000 7f7ec308c640 1 -- 172.21.15.5:0/244544282 shutdown_connections 2023-11-24T06:17:45.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.174+0000 7f7ec308c640 1 --2- 172.21.15.5:0/244544282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc06d9e0 0x7f7ebc165ef0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:45.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.174+0000 7f7ec308c640 1 -- 172.21.15.5:0/244544282 >> 172.21.15.5:0/244544282 conn(0x7f7ebc0cec10 msgr2=0x7f7ebc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:45.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.174+0000 7f7ec308c640 1 -- 172.21.15.5:0/244544282 shutdown_connections 2023-11-24T06:17:45.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.174+0000 7f7ec308c640 1 -- 172.21.15.5:0/244544282 wait complete. 2023-11-24T06:17:45.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.176+0000 7f7ec308c640 1 Processor -- start 2023-11-24T06:17:45.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.176+0000 7f7ec308c640 1 -- start start 2023-11-24T06:17:45.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.177+0000 7f7ec308c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ebc06d9e0 0x7f7ebc159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:45.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.177+0000 7f7ec308c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ebc144770 0x7f7ebc140390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:45.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.177+0000 7f7ec308c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc15f270 0x7f7ebc1408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:45.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.177+0000 7f7ec308c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc0ac970 con 0x7f7ebc15f270 2023-11-24T06:17:45.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.177+0000 7f7ec308c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc070130 con 0x7f7ebc144770 2023-11-24T06:17:45.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.177+0000 7f7ec308c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ebc070340 con 0x7f7ebc06d9e0 2023-11-24T06:17:45.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.177+0000 7f7ec288b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc15f270 0x7f7ebc1408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:45.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.177+0000 7f7ec288b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc15f270 0x7f7ebc1408d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50400/0 (socket says 172.21.15.5:50400) 2023-11-24T06:17:45.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.178+0000 7f7ec288b640 1 -- 172.21.15.5:0/2335047320 learned_addr learned my addr 172.21.15.5:0/2335047320 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:45.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.178+0000 7f7ec208a640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ebc06d9e0 0x7f7ebc159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:45.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.178+0000 7f7ec1889640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ebc144770 0x7f7ebc140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:45.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.178+0000 7f7ec288b640 1 -- 172.21.15.5:0/2335047320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ebc06d9e0 msgr2=0x7f7ebc159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:45.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.178+0000 7f7ec288b640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ebc06d9e0 0x7f7ebc159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:45.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.178+0000 7f7ec288b640 1 -- 172.21.15.5:0/2335047320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ebc144770 msgr2=0x7f7ebc140390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:45.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.178+0000 7f7ec288b640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ebc144770 0x7f7ebc140390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:45.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.178+0000 7f7ec288b640 1 -- 172.21.15.5:0/2335047320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb4009a70 con 0x7f7ebc15f270 2023-11-24T06:17:45.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.178+0000 7f7ec208a640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ebc06d9e0 0x7f7ebc159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:45.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.178+0000 7f7ec288b640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc15f270 0x7f7ebc1408d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009eb0 tx=0x7f7eb4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:45.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.179+0000 7f7eb2ffd640 1 -- 172.21.15.5:0/2335047320 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb403e610 con 0x7f7ebc15f270 2023-11-24T06:17:45.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.179+0000 7f7eb2ffd640 1 -- 172.21.15.5:0/2335047320 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eb403e770 con 0x7f7ebc15f270 2023-11-24T06:17:45.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.179+0000 7f7eb2ffd640 1 -- 172.21.15.5:0/2335047320 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb403eb20 con 0x7f7ebc15f270 2023-11-24T06:17:45.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.179+0000 7f7ec308c640 1 -- 172.21.15.5:0/2335047320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ebc070620 con 0x7f7ebc15f270 2023-11-24T06:17:45.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.179+0000 7f7ec308c640 1 -- 172.21.15.5:0/2335047320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ebc070be0 con 0x7f7ebc15f270 2023-11-24T06:17:45.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.180+0000 7f7ec308c640 1 -- 172.21.15.5:0/2335047320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc04e840 con 0x7f7ebc15f270 2023-11-24T06:17:45.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.181+0000 7f7eb2ffd640 1 -- 172.21.15.5:0/2335047320 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f7eb403b050 con 0x7f7ebc15f270 2023-11-24T06:17:45.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.182+0000 7f7eb2ffd640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7ea006aed0 0x7f7ea006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:45.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.182+0000 7f7eb2ffd640 1 -- 172.21.15.5:0/2335047320 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7eb40ab9e0 con 0x7f7ebc15f270 2023-11-24T06:17:45.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.182+0000 7f7ec208a640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7ea006aed0 0x7f7ea006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:45.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.182+0000 7f7ec208a640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7ea006aed0 0x7f7ea006d390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc143cc0 tx=0x7f7eb8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:45.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.186+0000 7f7eb2ffd640 1 -- 172.21.15.5:0/2335047320 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7eb4076a60 con 0x7f7ebc15f270 2023-11-24T06:17:45.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.423+0000 7f7ec308c640 1 -- 172.21.15.5:0/2335047320 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7ebc0b2460 con 0x7f7ebc15f270 2023-11-24T06:17:45.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.974+0000 7f7eb2ffd640 1 -- 172.21.15.5:0/2335047320 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7eb4009450 con 0x7f7ebc15f270 2023-11-24T06:17:45.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.978+0000 7f7ec308c640 1 -- 172.21.15.5:0/2335047320 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7ea006aed0 msgr2=0x7f7ea006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:45.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.978+0000 7f7ec308c640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7ea006aed0 0x7f7ea006d390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc143cc0 tx=0x7f7eb8009040 comp rx=0 tx=0).stop 2023-11-24T06:17:45.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.978+0000 7f7ec308c640 1 -- 172.21.15.5:0/2335047320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc15f270 msgr2=0x7f7ebc1408d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:45.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.978+0000 7f7ec308c640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc15f270 0x7f7ebc1408d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009eb0 tx=0x7f7eb4004a40 comp rx=0 tx=0).stop 2023-11-24T06:17:45.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.978+0000 7f7ec308c640 1 -- 172.21.15.5:0/2335047320 shutdown_connections 2023-11-24T06:17:45.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.978+0000 7f7ec308c640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f7ea006aed0 0x7f7ea006d390 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:45.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.978+0000 7f7ec308c640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ebc15f270 0x7f7ebc1408d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:45.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.978+0000 7f7ec308c640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f7ebc144770 0x7f7ebc140390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:45.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.978+0000 7f7ec308c640 1 --2- 172.21.15.5:0/2335047320 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f7ebc06d9e0 0x7f7ebc159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:45.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.978+0000 7f7ec308c640 1 -- 172.21.15.5:0/2335047320 >> 172.21.15.5:0/2335047320 conn(0x7f7ebc0cec10 msgr2=0x7f7ebc0cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:45.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.978+0000 7f7ec308c640 1 -- 172.21.15.5:0/2335047320 shutdown_connections 2023-11-24T06:17:45.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:45.979+0000 7f7ec308c640 1 -- 172.21.15.5:0/2335047320 wait complete. 2023-11-24T06:17:46.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T06:17:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.230+0000 7fe5849e6640 1 Processor -- start 2023-11-24T06:17:46.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.230+0000 7fe5849e6640 1 -- start start 2023-11-24T06:17:46.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.232+0000 7fe5849e6640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe58006dae0 0x7fe58015b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:46.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.232+0000 7fe5849e6640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe58015bbd0 0x7fe580070070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:46.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.232+0000 7fe5849e6640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe580144ca0 con 0x7fe5801448c0 2023-11-24T06:17:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.232+0000 7fe5849e6640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe58015c450 con 0x7fe58006dae0 2023-11-24T06:17:46.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.232+0000 7fe5849e6640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5800715b0 con 0x7fe58015bbd0 2023-11-24T06:17:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.233+0000 7fe57f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe5801448c0 0x7fe58006d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43636/0 (socket says 172.21.15.5:43636) 2023-11-24T06:17:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.233+0000 7fe57f7fe640 1 -- 172.21.15.5:0/3566643599 learned_addr learned my addr 172.21.15.5:0/3566643599 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.233+0000 7fe57effd640 1 --2- 172.21.15.5:0/3566643599 >> v2:172.21.15.6:3300/0 conn(0x7fe58006dae0 0x7fe58015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.233+0000 7fe57ffff640 1 --2- 172.21.15.5:0/3566643599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe58015bbd0 0x7fe580070070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.233+0000 7fe57e7fc640 1 -- 172.21.15.5:0/3566643599 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3329656949 0 0) 0x7fe580144ca0 con 0x7fe5801448c0 2023-11-24T06:17:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.233+0000 7fe57e7fc640 1 -- 172.21.15.5:0/3566643599 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe568003520 con 0x7fe5801448c0 2023-11-24T06:17:46.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.234+0000 7fe57effd640 1 -- 172.21.15.5:0/3566643599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe58015bbd0 msgr2=0x7fe580070070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.234+0000 7fe57effd640 1 --2- 172.21.15.5:0/3566643599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe58015bbd0 0x7fe580070070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.234+0000 7fe57e7fc640 1 -- 172.21.15.5:0/3566643599 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1390412827 0 0) 0x7fe568003520 con 0x7fe5801448c0 2023-11-24T06:17:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.234+0000 7fe57effd640 1 -- 172.21.15.5:0/3566643599 >> v1:172.21.15.5:6789/0 conn(0x7fe5801448c0 legacy=0x7fe58006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.234+0000 7fe57effd640 1 -- 172.21.15.5:0/3566643599 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe580071890 con 0x7fe58006dae0 2023-11-24T06:17:46.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.235+0000 7fe57effd640 1 --2- 172.21.15.5:0/3566643599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe58006dae0 0x7fe58015b690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe574009790 tx=0x7fe57402b770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bf7358c15decba4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.236+0000 7fe57e7fc640 1 -- 172.21.15.5:0/3566643599 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570001670 con 0x7fe58006dae0 2023-11-24T06:17:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.236+0000 7fe57e7fc640 1 -- 172.21.15.5:0/3566643599 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57402e050 con 0x7fe58006dae0 2023-11-24T06:17:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.236+0000 7fe57e7fc640 1 -- 172.21.15.5:0/3566643599 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe574030710 con 0x7fe58006dae0 2023-11-24T06:17:46.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.237+0000 7fe5849e6640 1 -- 172.21.15.5:0/3566643599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe58006dae0 msgr2=0x7fe58015b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:46.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.237+0000 7fe5849e6640 1 --2- 172.21.15.5:0/3566643599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe58006dae0 0x7fe58015b690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe574009790 tx=0x7fe57402b770 comp rx=0 tx=0).stop 2023-11-24T06:17:46.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.237+0000 7fe5849e6640 1 -- 172.21.15.5:0/3566643599 shutdown_connections 2023-11-24T06:17:46.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.237+0000 7fe5849e6640 1 --2- 172.21.15.5:0/3566643599 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe58015bbd0 0x7fe580070070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:46.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.237+0000 7fe5849e6640 1 --2- 172.21.15.5:0/3566643599 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe58006dae0 0x7fe58015b690 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:46.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.237+0000 7fe5849e6640 1 -- 172.21.15.5:0/3566643599 >> 172.21.15.5:0/3566643599 conn(0x7fe5800b0aa0 msgr2=0x7fe5800b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:46.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.237+0000 7fe5849e6640 1 -- 172.21.15.5:0/3566643599 shutdown_connections 2023-11-24T06:17:46.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.237+0000 7fe5849e6640 1 -- 172.21.15.5:0/3566643599 wait complete. 2023-11-24T06:17:46.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.239+0000 7fe5849e6640 1 Processor -- start 2023-11-24T06:17:46.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.239+0000 7fe5849e6640 1 -- start start 2023-11-24T06:17:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.240+0000 7fe5849e6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe58006dae0 0x7fe5800af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.240+0000 7fe5849e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5801448c0 0x7fe580143170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.240+0000 7fe5849e6640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe58015bbd0 0x7fe5801f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.240+0000 7fe5849e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5800ae820 con 0x7fe5801448c0 2023-11-24T06:17:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.240+0000 7fe5849e6640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5801f35d0 con 0x7fe58006dae0 2023-11-24T06:17:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.240+0000 7fe5849e6640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5801f38b0 con 0x7fe58015bbd0 2023-11-24T06:17:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.240+0000 7fe57effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5801448c0 0x7fe580143170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.240+0000 7fe57effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5801448c0 0x7fe580143170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50412/0 (socket says 172.21.15.5:50412) 2023-11-24T06:17:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.240+0000 7fe57effd640 1 -- 172.21.15.5:0/2622495584 learned_addr learned my addr 172.21.15.5:0/2622495584 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.241+0000 7fe57f7fe640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe58006dae0 0x7fe5800af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.241+0000 7fe57effd640 1 -- 172.21.15.5:0/2622495584 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe58015bbd0 msgr2=0x7fe5801f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.241+0000 7fe57ffff640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe58015bbd0 0x7fe5801f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.241+0000 7fe57effd640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe58015bbd0 0x7fe5801f2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.241+0000 7fe57effd640 1 -- 172.21.15.5:0/2622495584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe58006dae0 msgr2=0x7fe5800af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.241+0000 7fe57effd640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe58006dae0 0x7fe5800af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.241+0000 7fe57effd640 1 -- 172.21.15.5:0/2622495584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5740093a0 con 0x7fe5801448c0 2023-11-24T06:17:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.241+0000 7fe57f7fe640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe58006dae0 0x7fe5800af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.241+0000 7fe57effd640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5801448c0 0x7fe580143170 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe574000c00 tx=0x7fe5740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.241+0000 7fe567fff640 1 -- 172.21.15.5:0/2622495584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57403c820 con 0x7fe5801448c0 2023-11-24T06:17:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.241+0000 7fe567fff640 1 -- 172.21.15.5:0/2622495584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57403c980 con 0x7fe5801448c0 2023-11-24T06:17:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.241+0000 7fe567fff640 1 -- 172.21.15.5:0/2622495584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57403cd10 con 0x7fe5801448c0 2023-11-24T06:17:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.241+0000 7fe5849e6640 1 -- 172.21.15.5:0/2622495584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5800724f0 con 0x7fe5801448c0 2023-11-24T06:17:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.242+0000 7fe5849e6640 1 -- 172.21.15.5:0/2622495584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5800aed50 con 0x7fe5801448c0 2023-11-24T06:17:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.243+0000 7fe5849e6640 1 -- 172.21.15.5:0/2622495584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe58004e840 con 0x7fe5801448c0 2023-11-24T06:17:46.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.244+0000 7fe567fff640 1 -- 172.21.15.5:0/2622495584 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe574039050 con 0x7fe5801448c0 2023-11-24T06:17:46.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.244+0000 7fe567fff640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe56c06aed0 0x7fe56c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:46.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.245+0000 7fe567fff640 1 -- 172.21.15.5:0/2622495584 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe5740aaf30 con 0x7fe5801448c0 2023-11-24T06:17:46.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.245+0000 7fe57f7fe640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe56c06aed0 0x7fe56c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:46.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.246+0000 7fe57f7fe640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe56c06aed0 0x7fe56c06d390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe570001380 tx=0x7fe570009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:46.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.249+0000 7fe567fff640 1 -- 172.21.15.5:0/2622495584 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe574075e10 con 0x7fe5801448c0 2023-11-24T06:17:46.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.443+0000 7fe5849e6640 1 -- 172.21.15.5:0/2622495584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe5800aef10 con 0x7fe5801448c0 2023-11-24T06:17:46.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.993+0000 7fe567fff640 1 -- 172.21.15.5:0/2622495584 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe57404c0d0 con 0x7fe5801448c0 2023-11-24T06:17:46.997 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2023-11-24T06:17:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.999+0000 7fe5849e6640 1 -- 172.21.15.5:0/2622495584 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe56c06aed0 msgr2=0x7fe56c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.999+0000 7fe5849e6640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe56c06aed0 0x7fe56c06d390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe570001380 tx=0x7fe570009040 comp rx=0 tx=0).stop 2023-11-24T06:17:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.999+0000 7fe5849e6640 1 -- 172.21.15.5:0/2622495584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5801448c0 msgr2=0x7fe580143170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:46.999+0000 7fe5849e6640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5801448c0 0x7fe580143170 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe574000c00 tx=0x7fe5740046c0 comp rx=0 tx=0).stop 2023-11-24T06:17:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.000+0000 7fe5849e6640 1 -- 172.21.15.5:0/2622495584 shutdown_connections 2023-11-24T06:17:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.000+0000 7fe5849e6640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe56c06aed0 0x7fe56c06d390 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.000+0000 7fe5849e6640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe58015bbd0 0x7fe5801f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.000+0000 7fe5849e6640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5801448c0 0x7fe580143170 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.000+0000 7fe5849e6640 1 --2- 172.21.15.5:0/2622495584 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe58006dae0 0x7fe5800af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.000+0000 7fe5849e6640 1 -- 172.21.15.5:0/2622495584 >> 172.21.15.5:0/2622495584 conn(0x7fe5800b0aa0 msgr2=0x7fe5800ada50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.000+0000 7fe5849e6640 1 -- 172.21.15.5:0/2622495584 shutdown_connections 2023-11-24T06:17:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.000+0000 7fe5849e6640 1 -- 172.21.15.5:0/2622495584 wait complete. 2023-11-24T06:17:47.025 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T06:17:47.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.256+0000 7f0e0524b640 1 Processor -- start 2023-11-24T06:17:47.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.256+0000 7f0e0524b640 1 -- start start 2023-11-24T06:17:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.257+0000 7f0e0524b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0e0015d3b0 0x7f0e000ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.257+0000 7f0e0524b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0e000d0fa0 0x7f0e000d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:47.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.257+0000 7f0e0524b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e000c6bf0 con 0x7f0e000d1bc0 2023-11-24T06:17:47.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.257+0000 7f0e0524b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e000d1430 con 0x7f0e000d0fa0 2023-11-24T06:17:47.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.257+0000 7f0e0524b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e000d1fa0 con 0x7f0e0015d3b0 2023-11-24T06:17:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.258+0000 7f0dff7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0e000d0fa0 0x7f0e000d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.258+0000 7f0dff7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0e000d0fa0 0x7f0e000d1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50422/0 (socket says 172.21.15.5:50422) 2023-11-24T06:17:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.258+0000 7f0dff7fe640 1 -- 172.21.15.5:0/332011937 learned_addr learned my addr 172.21.15.5:0/332011937 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.258+0000 7f0e04a4a640 1 --1- 172.21.15.5:0/332011937 >> v1:172.21.15.5:6789/0 conn(0x7f0e000d1bc0 0x7f0e000b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43652/0 (socket says 172.21.15.5:43652) 2023-11-24T06:17:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.258+0000 7f0dfffff640 1 --2- 172.21.15.5:0/332011937 >> v2:172.21.15.6:3300/0 conn(0x7f0e0015d3b0 0x7f0e000ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.259+0000 7f0dff7fe640 1 -- 172.21.15.5:0/332011937 >> v2:172.21.15.6:3300/0 conn(0x7f0e0015d3b0 msgr2=0x7f0e000ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.259+0000 7f0dff7fe640 1 --2- 172.21.15.5:0/332011937 >> v2:172.21.15.6:3300/0 conn(0x7f0e0015d3b0 0x7f0e000ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.259+0000 7f0dff7fe640 1 -- 172.21.15.5:0/332011937 >> v1:172.21.15.5:6789/0 conn(0x7f0e000d1bc0 legacy=0x7f0e000b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.259+0000 7f0dff7fe640 1 -- 172.21.15.5:0/332011937 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e000cedc0 con 0x7f0e000d0fa0 2023-11-24T06:17:47.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.260+0000 7f0dff7fe640 1 --2- 172.21.15.5:0/332011937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e000d0fa0 0x7f0e000d1680 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0e0008b2a0 tx=0x7f0df402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277a909a216d464f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:47.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.260+0000 7f0dfeffd640 1 -- 172.21.15.5:0/332011937 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4031870 con 0x7f0e000d0fa0 2023-11-24T06:17:47.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.260+0000 7f0dfeffd640 1 -- 172.21.15.5:0/332011937 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df40319d0 con 0x7f0e000d0fa0 2023-11-24T06:17:47.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.260+0000 7f0dfeffd640 1 -- 172.21.15.5:0/332011937 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4031d60 con 0x7f0e000d0fa0 2023-11-24T06:17:47.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.261+0000 7f0e0524b640 1 -- 172.21.15.5:0/332011937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e000d0fa0 msgr2=0x7f0e000d1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:47.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.261+0000 7f0e0524b640 1 --2- 172.21.15.5:0/332011937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e000d0fa0 0x7f0e000d1680 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0e0008b2a0 tx=0x7f0df402b910 comp rx=0 tx=0).stop 2023-11-24T06:17:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.261+0000 7f0e0524b640 1 -- 172.21.15.5:0/332011937 shutdown_connections 2023-11-24T06:17:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.261+0000 7f0e0524b640 1 --2- 172.21.15.5:0/332011937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e000d0fa0 0x7f0e000d1680 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.261+0000 7f0e0524b640 1 --2- 172.21.15.5:0/332011937 >> v2:172.21.15.6:3300/0 conn(0x7f0e0015d3b0 0x7f0e000ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.261+0000 7f0e0524b640 1 -- 172.21.15.5:0/332011937 >> 172.21.15.5:0/332011937 conn(0x7f0e000d30a0 msgr2=0x7f0e000d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.261+0000 7f0e0524b640 1 -- 172.21.15.5:0/332011937 shutdown_connections 2023-11-24T06:17:47.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.261+0000 7f0e0524b640 1 -- 172.21.15.5:0/332011937 wait complete. 2023-11-24T06:17:47.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.263+0000 7f0e0524b640 1 Processor -- start 2023-11-24T06:17:47.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.263+0000 7f0e0524b640 1 -- start start 2023-11-24T06:17:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.264+0000 7f0e0524b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e000d0fa0 0x7f0e0015bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.264+0000 7f0e0524b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e000d1bc0 0x7f0e000cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.264+0000 7f0e0524b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e0015d3b0 0x7f0e000b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.264+0000 7f0e0524b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000cd200 con 0x7f0e000d0fa0 2023-11-24T06:17:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.264+0000 7f0e0524b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000cc580 con 0x7f0e000d1bc0 2023-11-24T06:17:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.264+0000 7f0e0524b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000cc860 con 0x7f0e0015d3b0 2023-11-24T06:17:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.264+0000 7f0dfffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e000d0fa0 0x7f0e0015bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:47.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.264+0000 7f0dfffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e000d0fa0 0x7f0e0015bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50432/0 (socket says 172.21.15.5:50432) 2023-11-24T06:17:47.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.264+0000 7f0dfffff640 1 -- 172.21.15.5:0/2401469618 learned_addr learned my addr 172.21.15.5:0/2401469618 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.265+0000 7f0dfffff640 1 -- 172.21.15.5:0/2401469618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e0015d3b0 msgr2=0x7f0e000b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.265+0000 7f0dfffff640 1 --2- 172.21.15.5:0/2401469618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e0015d3b0 0x7f0e000b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.265+0000 7f0dfffff640 1 -- 172.21.15.5:0/2401469618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e000d1bc0 msgr2=0x7f0e000cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.265+0000 7f0dff7fe640 1 --2- 172.21.15.5:0/2401469618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e000d1bc0 0x7f0e000cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.265+0000 7f0dfffff640 1 --2- 172.21.15.5:0/2401469618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e000d1bc0 0x7f0e000cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.265+0000 7f0dfffff640 1 -- 172.21.15.5:0/2401469618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df40094d0 con 0x7f0e000d0fa0 2023-11-24T06:17:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.265+0000 7f0dfffff640 1 --2- 172.21.15.5:0/2401469618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e000d0fa0 0x7f0e0015bb80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0df000d800 tx=0x7f0df000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.265+0000 7f0dfcff9640 1 -- 172.21.15.5:0/2401469618 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df000b6c0 con 0x7f0e000d0fa0 2023-11-24T06:17:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.265+0000 7f0dfcff9640 1 -- 172.21.15.5:0/2401469618 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df000b820 con 0x7f0e000d0fa0 2023-11-24T06:17:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.265+0000 7f0dfcff9640 1 -- 172.21.15.5:0/2401469618 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df000bbb0 con 0x7f0e000d0fa0 2023-11-24T06:17:47.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.265+0000 7f0e0524b640 1 -- 172.21.15.5:0/2401469618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e000ccb40 con 0x7f0e000d0fa0 2023-11-24T06:17:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.266+0000 7f0e0524b640 1 -- 172.21.15.5:0/2401469618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e001e65c0 con 0x7f0e000d0fa0 2023-11-24T06:17:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.266+0000 7f0e0524b640 1 -- 172.21.15.5:0/2401469618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e001f2f60 con 0x7f0e000d0fa0 2023-11-24T06:17:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.268+0000 7f0dfcff9640 1 -- 172.21.15.5:0/2401469618 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0df000bd10 con 0x7f0e000d0fa0 2023-11-24T06:17:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.269+0000 7f0dfcff9640 1 --2- 172.21.15.5:0/2401469618 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0dec06aed0 0x7f0dec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.269+0000 7f0dfcff9640 1 -- 172.21.15.5:0/2401469618 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f0df0089a80 con 0x7f0e000d0fa0 2023-11-24T06:17:47.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.269+0000 7f0dff7fe640 1 --2- 172.21.15.5:0/2401469618 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0dec06aed0 0x7f0dec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.270+0000 7f0dff7fe640 1 --2- 172.21.15.5:0/2401469618 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0dec06aed0 0x7f0dec06d390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0df4036000 tx=0x7f0df403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:47.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.273+0000 7f0dfcff9640 1 -- 172.21.15.5:0/2401469618 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0df0054ca0 con 0x7f0e000d0fa0 2023-11-24T06:17:47.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.466+0000 7f0e0524b640 1 -- 172.21.15.5:0/2401469618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0e0004e840 con 0x7f0e000d0fa0 2023-11-24T06:17:48.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:47.997+0000 7f0dfcff9640 1 -- 172.21.15.5:0/2401469618 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0df00582c0 con 0x7f0e000d0fa0 2023-11-24T06:17:48.001 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2023-11-24T06:17:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.001+0000 7f0e0524b640 1 -- 172.21.15.5:0/2401469618 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0dec06aed0 msgr2=0x7f0dec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.001+0000 7f0e0524b640 1 --2- 172.21.15.5:0/2401469618 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0dec06aed0 0x7f0dec06d390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0df4036000 tx=0x7f0df403c040 comp rx=0 tx=0).stop 2023-11-24T06:17:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.001+0000 7f0e0524b640 1 -- 172.21.15.5:0/2401469618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e000d0fa0 msgr2=0x7f0e0015bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.001+0000 7f0e0524b640 1 --2- 172.21.15.5:0/2401469618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e000d0fa0 0x7f0e0015bb80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0df000d800 tx=0x7f0df000dcc0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.001+0000 7f0e0524b640 1 -- 172.21.15.5:0/2401469618 shutdown_connections 2023-11-24T06:17:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.001+0000 7f0e0524b640 1 --2- 172.21.15.5:0/2401469618 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0dec06aed0 0x7f0dec06d390 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.001+0000 7f0e0524b640 1 --2- 172.21.15.5:0/2401469618 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0e0015d3b0 0x7f0e000b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.001+0000 7f0e0524b640 1 --2- 172.21.15.5:0/2401469618 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0e000d1bc0 0x7f0e000cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.001+0000 7f0e0524b640 1 --2- 172.21.15.5:0/2401469618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e000d0fa0 0x7f0e0015bb80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.001+0000 7f0e0524b640 1 -- 172.21.15.5:0/2401469618 >> 172.21.15.5:0/2401469618 conn(0x7f0e000d30a0 msgr2=0x7f0e001602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.001+0000 7f0e0524b640 1 -- 172.21.15.5:0/2401469618 shutdown_connections 2023-11-24T06:17:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.001+0000 7f0e0524b640 1 -- 172.21.15.5:0/2401469618 wait complete. 2023-11-24T06:17:48.030 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T06:17:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.244+0000 7f3391af1640 1 Processor -- start 2023-11-24T06:17:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.244+0000 7f3391af1640 1 -- start start 2023-11-24T06:17:48.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.245+0000 7f3391af1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f338c15af50 0x7f338c0b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:48.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.246+0000 7f3391af1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f338c0d13b0 0x7f338c15c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:48.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.246+0000 7f3391af1640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f338c0b13b0 con 0x7f338c0b2450 2023-11-24T06:17:48.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.246+0000 7f3391af1640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f338c0b11c0 con 0x7f338c15af50 2023-11-24T06:17:48.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.246+0000 7f3391af1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c15c640 con 0x7f338c0d13b0 2023-11-24T06:17:48.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.246+0000 7f3390aef640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f338c15af50 0x7f338c0b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:48.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.246+0000 7f3390aef640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f338c15af50 0x7f338c0b0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50436/0 (socket says 172.21.15.5:50436) 2023-11-24T06:17:48.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.246+0000 7f3390aef640 1 -- 172.21.15.5:0/3214248022 learned_addr learned my addr 172.21.15.5:0/3214248022 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:48.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.246+0000 7f338bfff640 1 --1- 172.21.15.5:0/3214248022 >> v1:172.21.15.5:6789/0 conn(0x7f338c0b2450 0x7f338c0d0ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43668/0 (socket says 172.21.15.5:43668) 2023-11-24T06:17:48.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.246+0000 7f33912f0640 1 --2- 172.21.15.5:0/3214248022 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f338c0d13b0 0x7f338c15c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.246+0000 7f3390aef640 1 -- 172.21.15.5:0/3214248022 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f338c0d13b0 msgr2=0x7f338c15c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.246+0000 7f3390aef640 1 --2- 172.21.15.5:0/3214248022 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f338c0d13b0 0x7f338c15c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.246+0000 7f3390aef640 1 -- 172.21.15.5:0/3214248022 >> v1:172.21.15.5:6789/0 conn(0x7f338c0b2450 legacy=0x7f338c0d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:48.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.246+0000 7f3390aef640 1 -- 172.21.15.5:0/3214248022 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c0cc830 con 0x7f338c15af50 2023-11-24T06:17:48.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.247+0000 7f33912f0640 1 --2- 172.21.15.5:0/3214248022 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f338c0d13b0 0x7f338c15c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:48.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.248+0000 7f3390aef640 1 --2- 172.21.15.5:0/3214248022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f338c15af50 0x7f338c0b0c80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f337c009e50 tx=0x7f337c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55868cc792f0267e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:48.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.248+0000 7f338b7fe640 1 -- 172.21.15.5:0/3214248022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c02cb00 con 0x7f338c15af50 2023-11-24T06:17:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.248+0000 7f338b7fe640 1 -- 172.21.15.5:0/3214248022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f337c02cc60 con 0x7f338c15af50 2023-11-24T06:17:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.248+0000 7f338b7fe640 1 -- 172.21.15.5:0/3214248022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f337c0343f0 con 0x7f338c15af50 2023-11-24T06:17:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.249+0000 7f3391af1640 1 -- 172.21.15.5:0/3214248022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f338c15af50 msgr2=0x7f338c0b0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:48.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.249+0000 7f3391af1640 1 --2- 172.21.15.5:0/3214248022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f338c15af50 0x7f338c0b0c80 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f337c009e50 tx=0x7f337c02bf60 comp rx=0 tx=0).stop 2023-11-24T06:17:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.249+0000 7f3391af1640 1 -- 172.21.15.5:0/3214248022 shutdown_connections 2023-11-24T06:17:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.249+0000 7f3391af1640 1 --2- 172.21.15.5:0/3214248022 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f338c0d13b0 0x7f338c15c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.249+0000 7f3391af1640 1 --2- 172.21.15.5:0/3214248022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f338c15af50 0x7f338c0b0c80 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.249+0000 7f3391af1640 1 -- 172.21.15.5:0/3214248022 >> 172.21.15.5:0/3214248022 conn(0x7f338c099a00 msgr2=0x7f338c099e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.249+0000 7f3391af1640 1 -- 172.21.15.5:0/3214248022 shutdown_connections 2023-11-24T06:17:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.249+0000 7f3391af1640 1 -- 172.21.15.5:0/3214248022 wait complete. 2023-11-24T06:17:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.251+0000 7f3391af1640 1 Processor -- start 2023-11-24T06:17:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.251+0000 7f3391af1640 1 -- start start 2023-11-24T06:17:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.253+0000 7f3391af1640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f338c0b2450 0x7f338c1598d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.253+0000 7f3391af1640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f338c0d13b0 0x7f338c0af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.253+0000 7f3391af1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f338c15af50 0x7f338c0af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.253+0000 7f3391af1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0c6140 con 0x7f338c15af50 2023-11-24T06:17:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.253+0000 7f3391af1640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0c6420 con 0x7f338c0b2450 2023-11-24T06:17:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.253+0000 7f3391af1640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338c0c6700 con 0x7f338c0d13b0 2023-11-24T06:17:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.253+0000 7f33912f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f338c15af50 0x7f338c0af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.253+0000 7f33912f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f338c15af50 0x7f338c0af9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50448/0 (socket says 172.21.15.5:50448) 2023-11-24T06:17:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.253+0000 7f33912f0640 1 -- 172.21.15.5:0/2123476923 learned_addr learned my addr 172.21.15.5:0/2123476923 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.253+0000 7f338bfff640 1 --2- 172.21.15.5:0/2123476923 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f338c0d13b0 0x7f338c0af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.253+0000 7f3390aef640 1 --2- 172.21.15.5:0/2123476923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f338c0b2450 0x7f338c1598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.254+0000 7f33912f0640 1 -- 172.21.15.5:0/2123476923 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f338c0d13b0 msgr2=0x7f338c0af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.254+0000 7f33912f0640 1 --2- 172.21.15.5:0/2123476923 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f338c0d13b0 0x7f338c0af4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.254+0000 7f33912f0640 1 -- 172.21.15.5:0/2123476923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f338c0b2450 msgr2=0x7f338c1598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.254+0000 7f33912f0640 1 --2- 172.21.15.5:0/2123476923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f338c0b2450 0x7f338c1598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.254+0000 7f33912f0640 1 -- 172.21.15.5:0/2123476923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f337c009b30 con 0x7f338c15af50 2023-11-24T06:17:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.254+0000 7f33912f0640 1 --2- 172.21.15.5:0/2123476923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f338c15af50 0x7f338c0af9e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f338400c7f0 tx=0x7f338400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.254+0000 7f33897fa640 1 -- 172.21.15.5:0/2123476923 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3384004220 con 0x7f338c15af50 2023-11-24T06:17:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.254+0000 7f33897fa640 1 -- 172.21.15.5:0/2123476923 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33840043a0 con 0x7f338c15af50 2023-11-24T06:17:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.254+0000 7f33897fa640 1 -- 172.21.15.5:0/2123476923 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3384007860 con 0x7f338c15af50 2023-11-24T06:17:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.254+0000 7f3391af1640 1 -- 172.21.15.5:0/2123476923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f338c0c69e0 con 0x7f338c15af50 2023-11-24T06:17:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.254+0000 7f3391af1640 1 -- 172.21.15.5:0/2123476923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f338c0c53c0 con 0x7f338c15af50 2023-11-24T06:17:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.256+0000 7f3391af1640 1 -- 172.21.15.5:0/2123476923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338c0c6c00 con 0x7f338c15af50 2023-11-24T06:17:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.256+0000 7f33897fa640 1 -- 172.21.15.5:0/2123476923 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f3384007bd0 con 0x7f338c15af50 2023-11-24T06:17:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.258+0000 7f33897fa640 1 --2- 172.21.15.5:0/2123476923 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f337806aed0 0x7f337806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.258+0000 7f33897fa640 1 -- 172.21.15.5:0/2123476923 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3384089420 con 0x7f338c15af50 2023-11-24T06:17:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.258+0000 7f3390aef640 1 --2- 172.21.15.5:0/2123476923 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f337806aed0 0x7f337806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.259+0000 7f3390aef640 1 --2- 172.21.15.5:0/2123476923 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f337806aed0 0x7f337806d390 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f337c035000 tx=0x7f337c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:48.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.263+0000 7f33897fa640 1 -- 172.21.15.5:0/2123476923 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f33840546d0 con 0x7f338c15af50 2023-11-24T06:17:48.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.467+0000 7f3391af1640 1 -- 172.21.15.5:0/2123476923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f338c159e10 con 0x7f338c15af50 2023-11-24T06:17:48.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.473+0000 7f33897fa640 1 -- 172.21.15.5:0/2123476923 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3384057cf0 con 0x7f338c15af50 2023-11-24T06:17:48.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.476+0000 7f3391af1640 1 -- 172.21.15.5:0/2123476923 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f337806aed0 msgr2=0x7f337806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:48.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.476+0000 7f3391af1640 1 --2- 172.21.15.5:0/2123476923 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f337806aed0 0x7f337806d390 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f337c035000 tx=0x7f337c030000 comp rx=0 tx=0).stop 2023-11-24T06:17:48.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.476+0000 7f3391af1640 1 -- 172.21.15.5:0/2123476923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f338c15af50 msgr2=0x7f338c0af9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:48.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.476+0000 7f3391af1640 1 --2- 172.21.15.5:0/2123476923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f338c15af50 0x7f338c0af9e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f338400c7f0 tx=0x7f338400ccb0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.476+0000 7f3391af1640 1 -- 172.21.15.5:0/2123476923 shutdown_connections 2023-11-24T06:17:48.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.476+0000 7f3391af1640 1 --2- 172.21.15.5:0/2123476923 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f337806aed0 0x7f337806d390 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.476+0000 7f3391af1640 1 --2- 172.21.15.5:0/2123476923 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f338c0d13b0 0x7f338c0af4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.476+0000 7f3391af1640 1 --2- 172.21.15.5:0/2123476923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f338c15af50 0x7f338c0af9e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.476+0000 7f3391af1640 1 --2- 172.21.15.5:0/2123476923 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f338c0b2450 0x7f338c1598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.476+0000 7f3391af1640 1 -- 172.21.15.5:0/2123476923 >> 172.21.15.5:0/2123476923 conn(0x7f338c099a00 msgr2=0x7f338c0cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:48.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.476+0000 7f3391af1640 1 -- 172.21.15.5:0/2123476923 shutdown_connections 2023-11-24T06:17:48.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.477+0000 7f3391af1640 1 -- 172.21.15.5:0/2123476923 wait complete. 2023-11-24T06:17:48.499 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_failed' 2023-11-24T06:17:48.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.721+0000 7fe50401c640 1 Processor -- start 2023-11-24T06:17:48.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.721+0000 7fe50401c640 1 -- start start 2023-11-24T06:17:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.722+0000 7fe50401c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4fc0ad720 0x7fe4fc0b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.722+0000 7fe50401c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe4fc0abff0 0x7fe4fc0b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.722+0000 7fe50401c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4fc15c360 con 0x7fe4fc06d520 2023-11-24T06:17:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.722+0000 7fe50401c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4fc0b41a0 con 0x7fe4fc0abff0 2023-11-24T06:17:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.722+0000 7fe50401c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc15ba30 con 0x7fe4fc0ad720 2023-11-24T06:17:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.722+0000 7fe5025b9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe4fc0abff0 0x7fe4fc0b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:48.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.722+0000 7fe5025b9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe4fc0abff0 0x7fe4fc0b25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50460/0 (socket says 172.21.15.5:50460) 2023-11-24T06:17:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.722+0000 7fe5025b9640 1 -- 172.21.15.5:0/3952921899 learned_addr learned my addr 172.21.15.5:0/3952921899 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.723+0000 7fe5015b7640 1 --2- 172.21.15.5:0/3952921899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4fc0ad720 0x7fe4fc0b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.723+0000 7fe5025b9640 1 -- 172.21.15.5:0/3952921899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4fc0ad720 msgr2=0x7fe4fc0b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.723+0000 7fe5025b9640 1 --2- 172.21.15.5:0/3952921899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4fc0ad720 0x7fe4fc0b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.723+0000 7fe5025b9640 1 -- 172.21.15.5:0/3952921899 >> v1:172.21.15.6:6789/0 conn(0x7fe4fc06d520 legacy=0x7fe4fc06d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.723+0000 7fe5025b9640 1 -- 172.21.15.5:0/3952921899 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc15bd10 con 0x7fe4fc0abff0 2023-11-24T06:17:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.725+0000 7fe5025b9640 1 --2- 172.21.15.5:0/3952921899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc0abff0 0x7fe4fc0b25c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009e50 tx=0x7fe4f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58a44051ac8ef70 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.725+0000 7fe500db6640 1 -- 172.21.15.5:0/3952921899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f402ca70 con 0x7fe4fc0abff0 2023-11-24T06:17:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.725+0000 7fe500db6640 1 -- 172.21.15.5:0/3952921899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f402cbd0 con 0x7fe4fc0abff0 2023-11-24T06:17:48.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.725+0000 7fe500db6640 1 -- 172.21.15.5:0/3952921899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40343f0 con 0x7fe4fc0abff0 2023-11-24T06:17:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.725+0000 7fe50401c640 1 -- 172.21.15.5:0/3952921899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc0abff0 msgr2=0x7fe4fc0b25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.725+0000 7fe50401c640 1 --2- 172.21.15.5:0/3952921899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc0abff0 0x7fe4fc0b25c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009e50 tx=0x7fe4f402bed0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.725+0000 7fe50401c640 1 -- 172.21.15.5:0/3952921899 shutdown_connections 2023-11-24T06:17:48.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.725+0000 7fe50401c640 1 --2- 172.21.15.5:0/3952921899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc0abff0 0x7fe4fc0b25c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.725+0000 7fe50401c640 1 --2- 172.21.15.5:0/3952921899 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4fc0ad720 0x7fe4fc0b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.725+0000 7fe50401c640 1 -- 172.21.15.5:0/3952921899 >> 172.21.15.5:0/3952921899 conn(0x7fe4fc0b0d80 msgr2=0x7fe4fc0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.725+0000 7fe50401c640 1 -- 172.21.15.5:0/3952921899 shutdown_connections 2023-11-24T06:17:48.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.725+0000 7fe50401c640 1 -- 172.21.15.5:0/3952921899 wait complete. 2023-11-24T06:17:48.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.727+0000 7fe50401c640 1 Processor -- start 2023-11-24T06:17:48.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.727+0000 7fe50401c640 1 -- start start 2023-11-24T06:17:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.728+0000 7fe50401c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4fc06d520 0x7fe4fc1f2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.728+0000 7fe50401c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4fc0abff0 0x7fe4fc1592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.728+0000 7fe50401c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc0ad720 0x7fe4fc0af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.728+0000 7fe50401c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc1f2a80 con 0x7fe4fc0ad720 2023-11-24T06:17:48.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.728+0000 7fe50401c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc15ae20 con 0x7fe4fc06d520 2023-11-24T06:17:48.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.728+0000 7fe50401c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc159d80 con 0x7fe4fc0abff0 2023-11-24T06:17:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.729+0000 7fe5025b9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc0ad720 0x7fe4fc0af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.729+0000 7fe5025b9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc0ad720 0x7fe4fc0af5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50474/0 (socket says 172.21.15.5:50474) 2023-11-24T06:17:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.729+0000 7fe5025b9640 1 -- 172.21.15.5:0/4243118872 learned_addr learned my addr 172.21.15.5:0/4243118872 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.729+0000 7fe501db8640 1 --2- 172.21.15.5:0/4243118872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4fc06d520 0x7fe4fc1f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:48.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.729+0000 7fe5015b7640 1 --2- 172.21.15.5:0/4243118872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4fc0abff0 0x7fe4fc1592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.729+0000 7fe5025b9640 1 -- 172.21.15.5:0/4243118872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4fc0abff0 msgr2=0x7fe4fc1592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.729+0000 7fe5025b9640 1 --2- 172.21.15.5:0/4243118872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4fc0abff0 0x7fe4fc1592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.729+0000 7fe5025b9640 1 -- 172.21.15.5:0/4243118872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4fc06d520 msgr2=0x7fe4fc1f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.729+0000 7fe5025b9640 1 --2- 172.21.15.5:0/4243118872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4fc06d520 0x7fe4fc1f2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.729+0000 7fe5025b9640 1 -- 172.21.15.5:0/4243118872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f4009b30 con 0x7fe4fc0ad720 2023-11-24T06:17:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.730+0000 7fe5025b9640 1 --2- 172.21.15.5:0/4243118872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc0ad720 0x7fe4fc0af5a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4036040 tx=0x7fe4f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.730+0000 7fe4f27fc640 1 -- 172.21.15.5:0/4243118872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4009510 con 0x7fe4fc0ad720 2023-11-24T06:17:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.730+0000 7fe4f27fc640 1 -- 172.21.15.5:0/4243118872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f403e450 con 0x7fe4fc0ad720 2023-11-24T06:17:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.730+0000 7fe4f27fc640 1 -- 172.21.15.5:0/4243118872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f403e7e0 con 0x7fe4fc0ad720 2023-11-24T06:17:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.730+0000 7fe50401c640 1 -- 172.21.15.5:0/4243118872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc15a060 con 0x7fe4fc0ad720 2023-11-24T06:17:48.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.730+0000 7fe50401c640 1 -- 172.21.15.5:0/4243118872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc15a620 con 0x7fe4fc0ad720 2023-11-24T06:17:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.732+0000 7fe50401c640 1 -- 172.21.15.5:0/4243118872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4fc074770 con 0x7fe4fc0ad720 2023-11-24T06:17:48.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.732+0000 7fe4f27fc640 1 -- 172.21.15.5:0/4243118872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe4f403e940 con 0x7fe4fc0ad720 2023-11-24T06:17:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.733+0000 7fe4f27fc640 1 --2- 172.21.15.5:0/4243118872 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe4e006aed0 0x7fe4e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.733+0000 7fe4f27fc640 1 -- 172.21.15.5:0/4243118872 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe4f40abf70 con 0x7fe4fc0ad720 2023-11-24T06:17:48.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.734+0000 7fe501db8640 1 --2- 172.21.15.5:0/4243118872 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe4e006aed0 0x7fe4e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:48.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.734+0000 7fe501db8640 1 --2- 172.21.15.5:0/4243118872 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe4e006aed0 0x7fe4e006d390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8004f60 tx=0x7fe4f8007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:48.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:48.738+0000 7fe4f27fc640 1 -- 172.21.15.5:0/4243118872 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4f40dee90 con 0x7fe4fc0ad720 2023-11-24T06:17:49.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:49.000+0000 7fe50401c640 1 -- 172.21.15.5:0/4243118872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_failed"]} v 0) v1 -- 0x7fe4fc0aef00 con 0x7fe4fc0ad720 2023-11-24T06:17:49.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:49.015+0000 7fe4f27fc640 1 -- 172.21.15.5:0/4243118872 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones.test_clone_failure_status_failed"]}]=0 v130) v1 ==== 165+0+0 (secure 0 0 0) 0x7fe4f40773f0 con 0x7fe4fc0ad720 2023-11-24T06:17:49.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:49.018+0000 7fe50401c640 1 -- 172.21.15.5:0/4243118872 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe4e006aed0 msgr2=0x7fe4e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:49.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:49.018+0000 7fe50401c640 1 --2- 172.21.15.5:0/4243118872 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe4e006aed0 0x7fe4e006d390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8004f60 tx=0x7fe4f8007e30 comp rx=0 tx=0).stop 2023-11-24T06:17:49.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:49.018+0000 7fe50401c640 1 -- 172.21.15.5:0/4243118872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc0ad720 msgr2=0x7fe4fc0af5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:49.018+0000 7fe50401c640 1 --2- 172.21.15.5:0/4243118872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc0ad720 0x7fe4fc0af5a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4036040 tx=0x7fe4f4004a40 comp rx=0 tx=0).stop 2023-11-24T06:17:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:49.019+0000 7fe50401c640 1 -- 172.21.15.5:0/4243118872 shutdown_connections 2023-11-24T06:17:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:49.019+0000 7fe50401c640 1 --2- 172.21.15.5:0/4243118872 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe4e006aed0 0x7fe4e006d390 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:49.019+0000 7fe50401c640 1 --2- 172.21.15.5:0/4243118872 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe4fc0abff0 0x7fe4fc1592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:49.019+0000 7fe50401c640 1 --2- 172.21.15.5:0/4243118872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4fc0ad720 0x7fe4fc0af5a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:49.019+0000 7fe50401c640 1 --2- 172.21.15.5:0/4243118872 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe4fc06d520 0x7fe4fc1f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:49.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:49.019+0000 7fe50401c640 1 -- 172.21.15.5:0/4243118872 >> 172.21.15.5:0/4243118872 conn(0x7fe4fc0b0d80 msgr2=0x7fe4fc0ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:49.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:49.019+0000 7fe50401c640 1 -- 172.21.15.5:0/4243118872 shutdown_connections 2023-11-24T06:17:49.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:49.019+0000 7fe50401c640 1 -- 172.21.15.5:0/4243118872 wait complete. 2023-11-24T06:17:49.045 INFO:tasks.cephfs_test_runner:ensure failure status is shown when clone is in failed state and validate the reason ... ERROR 2023-11-24T06:17:49.045 INFO:tasks.cephfs_test_runner: 2023-11-24T06:17:49.045 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T06:17:49.045 INFO:tasks.cephfs_test_runner:ERROR: test_clone_failure_status_failed (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2023-11-24T06:17:49.045 INFO:tasks.cephfs_test_runner:ensure failure status is shown when clone is in failed state and validate the reason 2023-11-24T06:17:49.045 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:17:49.045 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T06:17:49.045 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 6608, in test_clone_failure_status_failed 2023-11-24T06:17:49.046 INFO:tasks.cephfs_test_runner: self._do_subvolume_io(subvolume, number_of_files=200) 2023-11-24T06:17:49.046 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 278, in _do_subvolume_io 2023-11-24T06:17:49.046 INFO:tasks.cephfs_test_runner: filename = "{0}.{1}".format(TestVolumes.TEST_FILE_NAME_PREFIX, i) 2023-11-24T06:17:49.046 INFO:tasks.cephfs_test_runner:AttributeError: type object 'TestVolumes' has no attribute 'TEST_FILE_NAME_PREFIX' 2023-11-24T06:17:49.046 INFO:tasks.cephfs_test_runner: 2023-11-24T06:17:49.046 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:17:49.047 INFO:tasks.cephfs_test_runner:Ran 1 test in 76.159s 2023-11-24T06:17:49.047 INFO:tasks.cephfs_test_runner: 2023-11-24T06:17:49.047 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-24T06:17:49.047 INFO:tasks.cephfs_test_runner: 2023-11-24T06:17:49.047 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T06:17:49.047 INFO:tasks.cephfs_test_runner:ERROR: test_clone_failure_status_failed (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2023-11-24T06:17:49.047 INFO:tasks.cephfs_test_runner:ensure failure status is shown when clone is in failed state and validate the reason 2023-11-24T06:17:49.047 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T06:17:49.047 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T06:17:49.047 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 6608, in test_clone_failure_status_failed 2023-11-24T06:17:49.047 INFO:tasks.cephfs_test_runner: self._do_subvolume_io(subvolume, number_of_files=200) 2023-11-24T06:17:49.047 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs/test_volumes.py", line 278, in _do_subvolume_io 2023-11-24T06:17:49.048 INFO:tasks.cephfs_test_runner: filename = "{0}.{1}".format(TestVolumes.TEST_FILE_NAME_PREFIX, i) 2023-11-24T06:17:49.048 INFO:tasks.cephfs_test_runner:AttributeError: type object 'TestVolumes' has no attribute 'TEST_FILE_NAME_PREFIX' 2023-11-24T06:17:49.048 INFO:tasks.cephfs_test_runner: 2023-11-24T06:17:49.048 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_clone_failure_status_failed (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2023-11-24T06:17:49.117 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=3cff2a4b37ef43278fac27c72811efab Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_a13ea5ff00cb4df7a9cc941166ddeb358881144c/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_clone_failure_status_failed (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones) 2023-11-24T06:17:49.118 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-24T06:17:49.127 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-24T06:17:49.136 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-24T06:17:49.137 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:17:49.137 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:17:49.202 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:17:49.203 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-11-24T06:17:49.203 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:17:49.261 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T06:17:49.262 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:17:49.262 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:17:49.326 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:17:49.326 DEBUG:teuthology.orchestra.run.smithi006:> set -ex 2023-11-24T06:17:49.326 DEBUG:teuthology.orchestra.run.smithi006:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T06:17:49.376 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4055074,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3250028k,nr_inodes=812507,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6500064k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T06:17:49.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:49.377 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2023-11-24T06:17:49.440 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T06:17:49.440 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T06:17:49.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:49.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T06:17:49.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:49.547 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T06:17:49.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:49.638 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2023-11-24T06:17:49.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:49.802 DEBUG:teuthology.orchestra.run.smithi006:> ip netns list 2023-11-24T06:17:49.866 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T06:17:49.867 INFO:teuthology.orchestra.run.smithi006.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T06:17:49.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:49.867 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T06:17:49.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:49.975 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T06:17:50.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T06:17:50.077 DEBUG:teuthology.orchestra.run.smithi006:> sudo ip link delete ceph-brx 2023-11-24T06:17:50.231 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-24T06:17:50.242 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T06:17:50.243 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:17:50.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.477+0000 7ff92601a640 1 Processor -- start 2023-11-24T06:17:50.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.478+0000 7ff92601a640 1 -- start start 2023-11-24T06:17:50.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.479+0000 7ff92601a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff9200cf760 0x7ff92015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:50.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.479+0000 7ff92601a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff920143950 0x7ff92015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:50.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.479+0000 7ff92601a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff920071700 con 0x7ff920143570 2023-11-24T06:17:50.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.479+0000 7ff92601a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92015af30 con 0x7ff920143950 2023-11-24T06:17:50.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.479+0000 7ff92601a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92015b410 con 0x7ff9200cf760 2023-11-24T06:17:50.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.480+0000 7ff925819640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff920143950 0x7ff92015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:50.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.480+0000 7ff925819640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff920143950 0x7ff92015a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50476/0 (socket says 172.21.15.5:50476) 2023-11-24T06:17:50.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.480+0000 7ff925819640 1 -- 172.21.15.5:0/1626953657 learned_addr learned my addr 172.21.15.5:0/1626953657 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:50.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.480+0000 7ff924817640 1 --2- 172.21.15.5:0/1626953657 >> v2:172.21.15.6:3300/0 conn(0x7ff9200cf760 0x7ff92015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.480+0000 7ff925819640 1 -- 172.21.15.5:0/1626953657 >> v2:172.21.15.6:3300/0 conn(0x7ff9200cf760 msgr2=0x7ff92015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.480+0000 7ff925819640 1 --2- 172.21.15.5:0/1626953657 >> v2:172.21.15.6:3300/0 conn(0x7ff9200cf760 0x7ff92015a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.480+0000 7ff925819640 1 -- 172.21.15.5:0/1626953657 >> v1:172.21.15.6:6789/0 conn(0x7ff920143570 legacy=0x7ff9200cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.480+0000 7ff925819640 1 -- 172.21.15.5:0/1626953657 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92015b130 con 0x7ff920143950 2023-11-24T06:17:50.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.482+0000 7ff925819640 1 --2- 172.21.15.5:0/1626953657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff920143950 0x7ff92015a9f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff9200cbe10 tx=0x7ff91802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db682f23bc5bacb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:50.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.482+0000 7ff913fff640 1 -- 172.21.15.5:0/1626953657 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91802c9f0 con 0x7ff920143950 2023-11-24T06:17:50.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.482+0000 7ff913fff640 1 -- 172.21.15.5:0/1626953657 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91802cb50 con 0x7ff920143950 2023-11-24T06:17:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.482+0000 7ff913fff640 1 -- 172.21.15.5:0/1626953657 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9180343f0 con 0x7ff920143950 2023-11-24T06:17:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.483+0000 7ff92601a640 1 -- 172.21.15.5:0/1626953657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff920143950 msgr2=0x7ff92015a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.483+0000 7ff92601a640 1 --2- 172.21.15.5:0/1626953657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff920143950 0x7ff92015a9f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff9200cbe10 tx=0x7ff91802be50 comp rx=0 tx=0).stop 2023-11-24T06:17:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.483+0000 7ff92601a640 1 -- 172.21.15.5:0/1626953657 shutdown_connections 2023-11-24T06:17:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.483+0000 7ff92601a640 1 --2- 172.21.15.5:0/1626953657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff920143950 0x7ff92015a9f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.483+0000 7ff92601a640 1 --2- 172.21.15.5:0/1626953657 >> v2:172.21.15.6:3300/0 conn(0x7ff9200cf760 0x7ff92015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.483+0000 7ff92601a640 1 -- 172.21.15.5:0/1626953657 >> 172.21.15.5:0/1626953657 conn(0x7ff92006a860 msgr2=0x7ff92006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:50.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.483+0000 7ff92601a640 1 -- 172.21.15.5:0/1626953657 shutdown_connections 2023-11-24T06:17:50.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.483+0000 7ff92601a640 1 -- 172.21.15.5:0/1626953657 wait complete. 2023-11-24T06:17:50.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.485+0000 7ff92601a640 1 Processor -- start 2023-11-24T06:17:50.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.486+0000 7ff92601a640 1 -- start start 2023-11-24T06:17:50.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.487+0000 7ff92601a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9200cf760 0x7ff920159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:50.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.487+0000 7ff92601a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff920143570 0x7ff9200c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:50.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.487+0000 7ff92601a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff920143950 0x7ff92006c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:50.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.487+0000 7ff92601a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9201f4cf0 con 0x7ff920143570 2023-11-24T06:17:50.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.487+0000 7ff92601a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9201f3f30 con 0x7ff920143950 2023-11-24T06:17:50.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.487+0000 7ff92601a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9201f4210 con 0x7ff9200cf760 2023-11-24T06:17:50.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.487+0000 7ff924817640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff920143570 0x7ff9200c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:50.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.487+0000 7ff924817640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff920143570 0x7ff9200c50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50486/0 (socket says 172.21.15.5:50486) 2023-11-24T06:17:50.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.487+0000 7ff924817640 1 -- 172.21.15.5:0/3613488994 learned_addr learned my addr 172.21.15.5:0/3613488994 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.488+0000 7ff925018640 1 --2- 172.21.15.5:0/3613488994 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9200cf760 0x7ff920159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.488+0000 7ff925819640 1 --2- 172.21.15.5:0/3613488994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff920143950 0x7ff92006c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.488+0000 7ff924817640 1 -- 172.21.15.5:0/3613488994 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9200cf760 msgr2=0x7ff920159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.488+0000 7ff924817640 1 --2- 172.21.15.5:0/3613488994 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9200cf760 0x7ff920159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.488+0000 7ff924817640 1 -- 172.21.15.5:0/3613488994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff920143950 msgr2=0x7ff92006c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.488+0000 7ff924817640 1 --2- 172.21.15.5:0/3613488994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff920143950 0x7ff92006c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.488+0000 7ff924817640 1 -- 172.21.15.5:0/3613488994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff918009b30 con 0x7ff920143570 2023-11-24T06:17:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.488+0000 7ff924817640 1 --2- 172.21.15.5:0/3613488994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff920143570 0x7ff9200c50b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff91400d7d0 tx=0x7ff91400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.489+0000 7ff911ffb640 1 -- 172.21.15.5:0/3613488994 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91400b6c0 con 0x7ff920143570 2023-11-24T06:17:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.489+0000 7ff911ffb640 1 -- 172.21.15.5:0/3613488994 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91400b820 con 0x7ff920143570 2023-11-24T06:17:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.489+0000 7ff911ffb640 1 -- 172.21.15.5:0/3613488994 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91400bbb0 con 0x7ff920143570 2023-11-24T06:17:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.489+0000 7ff92601a640 1 -- 172.21.15.5:0/3613488994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9201f44f0 con 0x7ff920143570 2023-11-24T06:17:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.489+0000 7ff92601a640 1 -- 172.21.15.5:0/3613488994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9201599d0 con 0x7ff920143570 2023-11-24T06:17:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.490+0000 7ff92601a640 1 -- 172.21.15.5:0/3613488994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92004e840 con 0x7ff920143570 2023-11-24T06:17:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.491+0000 7ff911ffb640 1 -- 172.21.15.5:0/3613488994 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ff91400bd10 con 0x7ff920143570 2023-11-24T06:17:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.492+0000 7ff911ffb640 1 --2- 172.21.15.5:0/3613488994 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff90406aed0 0x7ff90406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.492+0000 7ff911ffb640 1 -- 172.21.15.5:0/3613488994 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff914089970 con 0x7ff920143570 2023-11-24T06:17:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.492+0000 7ff925018640 1 --2- 172.21.15.5:0/3613488994 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff90406aed0 0x7ff90406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:50.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.493+0000 7ff925018640 1 --2- 172.21.15.5:0/3613488994 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff90406aed0 0x7ff90406d390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff91c004ea0 tx=0x7ff91c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:50.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.496+0000 7ff911ffb640 1 -- 172.21.15.5:0/3613488994 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff914054cc0 con 0x7ff920143570 2023-11-24T06:17:50.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.687+0000 7ff92601a640 1 -- 172.21.15.5:0/3613488994 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff9201f49b0 con 0x7ff90406aed0 2023-11-24T06:17:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.690+0000 7ff911ffb640 1 -- 172.21.15.5:0/3613488994 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48633 (secure 0 0 0) 0x7ff9201f49b0 con 0x7ff90406aed0 2023-11-24T06:17:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-11-24T06:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.692+0000 7ff92601a640 1 -- 172.21.15.5:0/3613488994 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff90406aed0 msgr2=0x7ff90406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.692+0000 7ff92601a640 1 --2- 172.21.15.5:0/3613488994 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff90406aed0 0x7ff90406d390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff91c004ea0 tx=0x7ff91c009040 comp rx=0 tx=0).stop 2023-11-24T06:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.693+0000 7ff92601a640 1 -- 172.21.15.5:0/3613488994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff920143570 msgr2=0x7ff9200c50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.693+0000 7ff92601a640 1 --2- 172.21.15.5:0/3613488994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff920143570 0x7ff9200c50b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff91400d7d0 tx=0x7ff91400dc90 comp rx=0 tx=0).stop 2023-11-24T06:17:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.693+0000 7ff92601a640 1 -- 172.21.15.5:0/3613488994 shutdown_connections 2023-11-24T06:17:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.693+0000 7ff92601a640 1 --2- 172.21.15.5:0/3613488994 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff90406aed0 0x7ff90406d390 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.693+0000 7ff92601a640 1 --2- 172.21.15.5:0/3613488994 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff920143950 0x7ff92006c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.693+0000 7ff92601a640 1 --2- 172.21.15.5:0/3613488994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff920143570 0x7ff9200c50b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.693+0000 7ff92601a640 1 --2- 172.21.15.5:0/3613488994 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff9200cf760 0x7ff920159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.693+0000 7ff92601a640 1 -- 172.21.15.5:0/3613488994 >> 172.21.15.5:0/3613488994 conn(0x7ff92006a860 msgr2=0x7ff92006ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.693+0000 7ff92601a640 1 -- 172.21.15.5:0/3613488994 shutdown_connections 2023-11-24T06:17:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.693+0000 7ff92601a640 1 -- 172.21.15.5:0/3613488994 wait complete. 2023-11-24T06:17:50.730 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-11-24T06:17:50.124916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":241452,"kb_used_data":4472,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733268,"statfs":{"total":773094113280,"available":772846866432,"internally_reserved":0,"allocated":4579328,"data_stored":2538553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33927,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001779"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":175,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391587+0000","last_change":"2023-11-24T06:15:42.021608+0000","last_active":"2023-11-24T06:17:18.391587+0000","last_peered":"2023-11-24T06:17:18.391587+0000","last_clean":"2023-11-24T06:17:18.391587+0000","last_became_active":"2023-11-24T06:15:42.018847+0000","last_became_peered":"2023-11-24T06:15:42.018847+0000","last_unstale":"2023-11-24T06:17:18.391587+0000","last_undegraded":"2023-11-24T06:17:18.391587+0000","last_fullsized":"2023-11-24T06:17:18.391587+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_clean_scrub_stamp":"2023-11-24T06:15:40.694368+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:41.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509467,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":987136,"data_stored":744396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2370000000000001,"15min":1.2509999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.663,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.238,"5min":1.2509999999999999,"15min":1.2529999999999999},"min":{"1min":0.70899999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7130000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.004}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.2250000000000001,"15min":1.2330000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.3700000000000001,"15min":1.3879999999999999},"min":{"1min":0.85399999999999998,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6830000000000001,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":1.226}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.4650000000000001,"15min":1.496},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.5980000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4930000000000001,"15min":1.51},"min":{"1min":0.86799999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.8280000000000001,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.5229999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.444,"15min":1.464},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.923,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.6519999999999999,"15min":1.6819999999999999},"min":{"1min":0.92200000000000004,"5min":0.879,"15min":0.879},"max":{"1min":1.857,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":1.3740000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.774,"15min":1.8180000000000001},"min":{"1min":1.081,"5min":1.016,"15min":1.016},"max":{"1min":2.141,"5min":5.7560000000000002,"15min":5.7560000000000002},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.581,"5min":1.669,"15min":1.6839999999999999},"min":{"1min":1.26,"5min":1.018,"15min":1.018},"max":{"1min":2.0910000000000002,"5min":2.681,"15min":2.681},"last":1.6850000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":2.0299999999999998,"15min":2.0880000000000001},"min":{"1min":1.0760000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.0960000000000001,"5min":6.6260000000000003,"15min":6.6260000000000003},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.663,"5min":2.0750000000000002,"15min":2.1440000000000001},"min":{"1min":1.155,"5min":1.155,"15min":1.155},"max":{"1min":2.0459999999999998,"5min":6.4169999999999998,"15min":6.4169999999999998},"last":1.155}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.7749999999999999,"5min":1.972,"15min":2.0049999999999999},"min":{"1min":1.214,"5min":1.214,"15min":1.214},"max":{"1min":2.198,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.633,"5min":1.853,"15min":1.889},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":2.173,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":1.587}]}]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":160123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.5449999999999999,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.0980000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.569,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.569}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.2370000000000001,"15min":1.246},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.77,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.266,"15min":1.2769999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.722,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.0940000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.246,"15min":1.2509999999999999},"min":{"1min":0.75700000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.994,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.4039999999999999,"15min":1.4199999999999999},"min":{"1min":0.84899999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.8029999999999999,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.4059999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.337,"15min":1.349},"min":{"1min":0.84099999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.796,"5min":2.331,"15min":2.331},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.53,"15min":1.55},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":1.3149999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.585,"15min":1.615},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.901,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.5229999999999999,"15min":1.5489999999999999},"min":{"1min":0.88600000000000001,"5min":0.873,"15min":0.873},"max":{"1min":1.9339999999999999,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.7450000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.6759999999999999,"15min":1.6930000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.464},{"interface":"front","average":{"1min":1.627,"5min":1.738,"15min":1.756},"min":{"1min":1.141,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":2.016,"5min":2.802,"15min":2.802},"last":1.6919999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.748,"15min":1.774},"min":{"1min":1.1399999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.048,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":1.851},{"interface":"front","average":{"1min":1.679,"5min":1.7549999999999999,"15min":1.7669999999999999},"min":{"1min":1.087,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.651}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":166022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.1679999999999999,"15min":1.204},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.482,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.72599999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6040000000000001,"5min":1.875,"15min":1.875},"last":0.98999999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.169,"15min":1.175},"min":{"1min":0.79300000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.609,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.802},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.282,"15min":1.3200000000000001},"min":{"1min":0.69499999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.526,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.24}]},{"osd":2,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.147,"15min":1.167},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.343,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.4690000000000001,"15min":1.5149999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.6399999999999999,"5min":2.633,"15min":2.633},"last":1.74}]},{"osd":3,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.3660000000000001,"15min":1.3939999999999999},"min":{"1min":0.85499999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5249999999999999,"5min":2.323,"15min":2.323},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.544,"15min":1.5820000000000001},"min":{"1min":0.90500000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.694,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":1.3109999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.5620000000000001,"15min":1.5900000000000001},"min":{"1min":0.64500000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7569999999999999,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":2.1219999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.49,"15min":1.5189999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.8899999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.1319999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.635,"15min":1.663},"min":{"1min":0.83499999999999996,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.845,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":1.401},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.579,"15min":1.627},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6890000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.6899999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.738,"15min":1.7709999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.0230000000000001,"5min":2.6880000000000002,"15min":2.6880000000000002},"last":1.502},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.6970000000000001,"15min":1.74},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.994,"5min":2.448,"15min":2.448},"last":1.7689999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0489999999999999,"15min":1.0369999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1220000000000001,"15min":1.1200000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.28}]},{"osd":1,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2529999999999999,"15min":1.2689999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.861,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.256,"15min":1.254},"min":{"1min":0.81100000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8200000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.6000000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.71899999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.55,"5min":1.677,"15min":1.677},"last":1.502},{"interface":"front","average":{"1min":1.359,"5min":1.419,"15min":1.429},"min":{"1min":0.72099999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.8520000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.3320000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2969999999999999,"15min":1.3029999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.655,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.5169999999999999,"15min":1.5269999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9450000000000001,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":1.45}]},{"osd":4,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.556,"15min":1.5580000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.95,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.8},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.82799999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.1899999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":1.7729999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.575,"15min":1.571},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.298,"5min":2.6480000000000001,"15min":2.6480000000000001},"last":2.109},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.6060000000000001,"15min":1.6220000000000001},"min":{"1min":1.028,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.887,"5min":3.1560000000000001,"15min":3.1560000000000001},"last":1.387}]},{"osd":7,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.655,"15min":1.657},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.23,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.6599999999999999,"5min":1.782,"15min":1.802},"min":{"1min":0.80100000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.1760000000000002,"5min":4.1779999999999999,"15min":4.1779999999999999},"last":1.728}]}]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":450560,"data_stored":197451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.93400000000000005,"15min":0.95099999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2210000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.03,"15min":1.044},"min":{"1min":0.48299999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.431,"5min":1.518,"15min":1.518},"last":1.1930000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.236,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.083,"15min":1.1259999999999999},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0980000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.3360000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.151,"15min":1.1870000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.603,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.405,"5min":1.875,"15min":1.875},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2310000000000001,"15min":1.2509999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.4770000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.022},{"interface":"front","average":{"1min":1.202,"5min":1.4370000000000001,"15min":1.476},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.776,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.5249999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.3480000000000001,"15min":1.375},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5660000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.137},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.5189999999999999,"15min":1.552},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.865,"5min":3.0089999999999999,"15min":3.0089999999999999},"last":1.5900000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.4319999999999999,"15min":1.4490000000000001},"min":{"1min":0.96699999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.8360000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.5,"15min":1.5229999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.944,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.3720000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.532,"15min":1.5589999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.9159999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.5780000000000001,"15min":1.609},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.869,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.546}]}]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31608,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.96099999999999997,"15min":0.94899999999999995},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.2709999999999999,"5min":5.2709999999999999,"15min":5.2709999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.157,"5min":1.079,"15min":1.0660000000000001},"min":{"1min":0.47499999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.5650000000000004,"5min":5.5650000000000004,"15min":5.5650000000000004},"last":1.2110000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1,"15min":0.99199999999999999},"min":{"1min":0.45700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":5.7140000000000004,"5min":5.7140000000000004,"15min":5.7140000000000004},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.0529999999999999,"15min":1.079},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.444,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":1.3759999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.131,"15min":1.1679999999999999},"min":{"1min":0.59599999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.403,"5min":1.95,"15min":1.95},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1839999999999999,"15min":1.1799999999999999},"min":{"1min":0.57699999999999996,"5min":0.504,"15min":0.504},"max":{"1min":5.4820000000000002,"5min":5.4820000000000002,"15min":5.4820000000000002},"last":1.466}]},{"osd":4,"last update":"Fri Nov 24 06:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.2789999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":4.1280000000000001,"5min":4.1280000000000001,"15min":4.1280000000000001},"last":1.121},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.4319999999999999,"15min":1.46},"min":{"1min":0.83099999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.7649999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.631}]},{"osd":6,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.427,"15min":1.4370000000000001},"min":{"1min":0.76700000000000002,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.6440000000000001,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":1.8120000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.357,"15min":1.383},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.2120000000000002,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.4379999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.681}]}]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":421888,"data_stored":166151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.89500000000000002,"15min":0.89100000000000001},"min":{"1min":0.66600000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.393,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.83599999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.91100000000000003,"15min":0.90100000000000002},"min":{"1min":0.72399999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3340000000000001,"5min":1.919,"15min":1.919},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.121,"15min":1.1120000000000001},"min":{"1min":0.71599999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6719999999999999,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.6719999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.0940000000000001,"15min":1.077},"min":{"1min":0.56399999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5940000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.579},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.004,"15min":0.998},"min":{"1min":0.54900000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.746,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":1.222}]},{"osd":4,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.3480000000000001,"15min":1.3340000000000001},"min":{"1min":1.2490000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7270000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.484},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3009999999999999,"15min":1.3049999999999999},"min":{"1min":0.90100000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.6579999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.542}]},{"osd":5,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.304,"15min":1.2909999999999999},"min":{"1min":0.88200000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.982,"5min":2.085,"15min":2.085},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3959999999999999,"15min":1.397},"min":{"1min":0.94499999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.6279999999999999,"5min":2.355,"15min":2.355},"last":1.2729999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.421,"15min":1.4159999999999999},"min":{"1min":1.046,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.944,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.4199999999999999,"15min":1.4119999999999999},"min":{"1min":0.96999999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8799999999999999,"5min":2.298,"15min":2.298},"last":1.3100000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.544,"15min":1.5389999999999999},"min":{"1min":1.218,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.0299999999999998,"5min":2.492,"15min":2.492},"last":1.383},{"interface":"front","average":{"1min":1.53,"5min":1.482,"15min":1.474},"min":{"1min":0.91400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.0830000000000002,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.7230000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1024000,"data_stored":761236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.89300000000000002,"15min":0.91900000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.127,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.95499999999999996,"15min":0.97499999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5600000000000001,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.4850000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.91100000000000003,"15min":0.92200000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3680000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.048,"15min":1.0549999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.6699999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":1.167}]},{"osd":3,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.996,"15min":1.01},"min":{"1min":0.495,"5min":0.434,"15min":0.434},"max":{"1min":1.444,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.999,"5min":1.0980000000000001,"15min":1.1140000000000001},"min":{"1min":0.59099999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.522,"5min":2.653,"15min":2.653},"last":1.266}]},{"osd":4,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3320000000000001,"15min":1.347},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.8260000000000001,"5min":2.254,"15min":2.254},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3740000000000001,"15min":1.3939999999999999},"min":{"1min":0.84499999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8620000000000001,"5min":1.996,"15min":1.996},"last":1.421}]},{"osd":5,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.4259999999999999,"15min":1.4399999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.956,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.744},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.4410000000000001,"15min":1.4690000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.919,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.623}]},{"osd":6,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.5029999999999999,"15min":1.518},"min":{"1min":0.90200000000000002,"5min":0.746,"15min":0.746},"max":{"1min":2.0939999999999999,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.494,"15min":1.52},"min":{"1min":0.78100000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.891,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.8680000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.5429999999999999,"15min":1.5509999999999999},"min":{"1min":0.89200000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.228},{"interface":"front","average":{"1min":1.359,"5min":1.4990000000000001,"15min":1.522},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.925,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.7709999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:17:50.732 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:17:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.955+0000 7f659bd89640 1 Processor -- start 2023-11-24T06:17:50.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.955+0000 7f659bd89640 1 -- start start 2023-11-24T06:17:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.956+0000 7f659bd89640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6594161930 0x7f65940ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.957+0000 7f659bd89640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65940ad510 con 0x7f65940719c0 2023-11-24T06:17:50.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.957+0000 7f659bd89640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65940acc50 con 0x7f659406d520 2023-11-24T06:17:50.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.957+0000 7f659bd89640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6594071e60 con 0x7f6594161930 2023-11-24T06:17:50.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.957+0000 7f6599324640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f65940719c0 0x7f65940ad9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43680/0 (socket says 172.21.15.5:43680) 2023-11-24T06:17:50.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.957+0000 7f6599324640 1 -- 172.21.15.5:0/2449726376 learned_addr learned my addr 172.21.15.5:0/2449726376 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:50.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.957+0000 7f6599b25640 1 --2- 172.21.15.5:0/2449726376 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6594161930 0x7f65940ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:50.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.957+0000 7f6598b23640 1 -- 172.21.15.5:0/2449726376 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1627752568 0 0) 0x7f65940ad510 con 0x7f65940719c0 2023-11-24T06:17:50.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.958+0000 7f6598b23640 1 -- 172.21.15.5:0/2449726376 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6580003520 con 0x7f65940719c0 2023-11-24T06:17:50.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.958+0000 7f6598b23640 1 -- 172.21.15.5:0/2449726376 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 98559714 0 0) 0x7f6580003520 con 0x7f65940719c0 2023-11-24T06:17:50.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.958+0000 7f6598b23640 1 -- 172.21.15.5:0/2449726376 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6594161930 msgr2=0x7f65940ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:50.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.958+0000 7f6598b23640 1 --2- 172.21.15.5:0/2449726376 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6594161930 0x7f65940ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.958+0000 7f6598b23640 1 -- 172.21.15.5:0/2449726376 >> v1:172.21.15.6:6789/0 conn(0x7f659406d520 legacy=0x7f65940b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:50.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.958+0000 7f6598b23640 1 -- 172.21.15.5:0/2449726376 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65940acec0 con 0x7f65940719c0 2023-11-24T06:17:50.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.958+0000 7f6598b23640 1 -- 172.21.15.5:0/2449726376 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f6584001670 con 0x7f65940719c0 2023-11-24T06:17:50.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.958+0000 7f6598b23640 1 -- 172.21.15.5:0/2449726376 >> v1:172.21.15.5:6789/0 conn(0x7f65940719c0 legacy=0x7f65940ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:50.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.958+0000 7f6598b23640 1 --2- 172.21.15.5:0/2449726376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65800039a0 0x7f6580003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:50.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.958+0000 7f6598b23640 1 --2- 172.21.15.5:0/2449726376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65800047c0 0x7f6580004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:50.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.958+0000 7f6598b23640 1 --2- 172.21.15.5:0/2449726376 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6594161930 0x7f6580005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:50.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.958+0000 7f6598b23640 1 -- 172.21.15.5:0/2449726376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6594071e60 con 0x7f65800039a0 2023-11-24T06:17:50.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.958+0000 7f6598b23640 1 -- 172.21.15.5:0/2449726376 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658000ba60 con 0x7f65800047c0 2023-11-24T06:17:50.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.958+0000 7f6598b23640 1 -- 172.21.15.5:0/2449726376 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658000bc20 con 0x7f6594161930 2023-11-24T06:17:50.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.959+0000 7f6599b25640 1 --2- 172.21.15.5:0/2449726376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65800039a0 0x7f6580003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:50.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.959+0000 7f6599324640 1 --2- 172.21.15.5:0/2449726376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65800047c0 0x7f6580004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:50.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.959+0000 7f6599b25640 1 -- 172.21.15.5:0/2449726376 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6594161930 msgr2=0x7f6580005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:50.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.959+0000 7f6599b25640 1 --2- 172.21.15.5:0/2449726376 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f6594161930 0x7f6580005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.959+0000 7f6599b25640 1 -- 172.21.15.5:0/2449726376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65800047c0 msgr2=0x7f6580004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:50.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.959+0000 7f6599b25640 1 --2- 172.21.15.5:0/2449726376 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f65800047c0 0x7f6580004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.959+0000 7f6599b25640 1 -- 172.21.15.5:0/2449726376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6580003520 con 0x7f65800039a0 2023-11-24T06:17:50.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.961+0000 7f6599b25640 1 --2- 172.21.15.5:0/2449726376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65800039a0 0x7f6580003dc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6590009a90 tx=0x7f65800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2581e852523a5e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:50.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.961+0000 7f6598b23640 1 -- 172.21.15.5:0/2449726376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659002c690 con 0x7f65800039a0 2023-11-24T06:17:50.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.961+0000 7f6598b23640 1 -- 172.21.15.5:0/2449726376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659002c7f0 con 0x7f65800039a0 2023-11-24T06:17:50.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.961+0000 7f6598b23640 1 -- 172.21.15.5:0/2449726376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659002cb80 con 0x7f65800039a0 2023-11-24T06:17:50.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.962+0000 7f659bd89640 1 -- 172.21.15.5:0/2449726376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65800039a0 msgr2=0x7f6580003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:50.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.962+0000 7f659bd89640 1 --2- 172.21.15.5:0/2449726376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65800039a0 0x7f6580003dc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6590009a90 tx=0x7f65800057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.962+0000 7f659a326640 1 -- 172.21.15.5:0/2449726376 reap_dead start 2023-11-24T06:17:50.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.962+0000 7f659bd89640 1 -- 172.21.15.5:0/2449726376 shutdown_connections 2023-11-24T06:17:50.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.962+0000 7f659bd89640 1 -- 172.21.15.5:0/2449726376 >> 172.21.15.5:0/2449726376 conn(0x7f65940b0d80 msgr2=0x7f65940b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:50.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.962+0000 7f659bd89640 1 -- 172.21.15.5:0/2449726376 shutdown_connections 2023-11-24T06:17:50.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.962+0000 7f659bd89640 1 -- 172.21.15.5:0/2449726376 wait complete. 2023-11-24T06:17:50.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.964+0000 7f659bd89640 1 Processor -- start 2023-11-24T06:17:50.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.964+0000 7f659bd89640 1 -- start start 2023-11-24T06:17:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.965+0000 7f659bd89640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65800039a0 0x7f659415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.965+0000 7f659bd89640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65940cccd0 0x7f6594150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.965+0000 7f659bd89640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f659415e760 0x7f659415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.965+0000 7f659bd89640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65940cd400 con 0x7f65800039a0 2023-11-24T06:17:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.965+0000 7f659bd89640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659415f120 con 0x7f659415e760 2023-11-24T06:17:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.965+0000 7f659bd89640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6594099170 con 0x7f65940cccd0 2023-11-24T06:17:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.965+0000 7f6599b25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65800039a0 0x7f659415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:50.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.965+0000 7f6599b25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65800039a0 0x7f659415dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50504/0 (socket says 172.21.15.5:50504) 2023-11-24T06:17:50.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.965+0000 7f6599b25640 1 -- 172.21.15.5:0/152681571 learned_addr learned my addr 172.21.15.5:0/152681571 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:50.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.965+0000 7f6599324640 1 --2- 172.21.15.5:0/152681571 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65940cccd0 0x7f6594150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:50.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.965+0000 7f659a326640 1 --2- 172.21.15.5:0/152681571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f659415e760 0x7f659415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:50.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.965+0000 7f6599b25640 1 -- 172.21.15.5:0/152681571 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65940cccd0 msgr2=0x7f6594150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:50.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.966+0000 7f6599b25640 1 --2- 172.21.15.5:0/152681571 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65940cccd0 0x7f6594150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.966+0000 7f6599b25640 1 -- 172.21.15.5:0/152681571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f659415e760 msgr2=0x7f659415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:50.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.966+0000 7f6599b25640 1 --2- 172.21.15.5:0/152681571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f659415e760 0x7f659415ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:50.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.966+0000 7f6599b25640 1 -- 172.21.15.5:0/152681571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65940cd130 con 0x7f65800039a0 2023-11-24T06:17:50.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.966+0000 7f6599b25640 1 --2- 172.21.15.5:0/152681571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65800039a0 0x7f659415dbc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f659002c230 tx=0x7f6590004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:50.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.966+0000 7f658a7fc640 1 -- 172.21.15.5:0/152681571 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590003810 con 0x7f65800039a0 2023-11-24T06:17:50.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.966+0000 7f658a7fc640 1 -- 172.21.15.5:0/152681571 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6590003970 con 0x7f65800039a0 2023-11-24T06:17:50.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.966+0000 7f659bd89640 1 -- 172.21.15.5:0/152681571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659415c1b0 con 0x7f65800039a0 2023-11-24T06:17:50.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.966+0000 7f658a7fc640 1 -- 172.21.15.5:0/152681571 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6590003ce0 con 0x7f65800039a0 2023-11-24T06:17:50.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.967+0000 7f659bd89640 1 -- 172.21.15.5:0/152681571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659415c770 con 0x7f65800039a0 2023-11-24T06:17:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.968+0000 7f659bd89640 1 -- 172.21.15.5:0/152681571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65940cd550 con 0x7f65800039a0 2023-11-24T06:17:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.969+0000 7f658a7fc640 1 -- 172.21.15.5:0/152681571 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6590040450 con 0x7f65800039a0 2023-11-24T06:17:50.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.970+0000 7f658a7fc640 1 --2- 172.21.15.5:0/152681571 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f657806aed0 0x7f657806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.970+0000 7f658a7fc640 1 -- 172.21.15.5:0/152681571 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f65900ab0e0 con 0x7f65800039a0 2023-11-24T06:17:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.970+0000 7f6599324640 1 --2- 172.21.15.5:0/152681571 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f657806aed0 0x7f657806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.971+0000 7f6599324640 1 --2- 172.21.15.5:0/152681571 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f657806aed0 0x7f657806d390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6584001eb0 tx=0x7f6584000f70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:50.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:50.974+0000 7f658a7fc640 1 -- 172.21.15.5:0/152681571 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65900764a0 con 0x7f65800039a0 2023-11-24T06:17:51.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.168+0000 7f659bd89640 1 -- 172.21.15.5:0/152681571 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6594070c90 con 0x7f657806aed0 2023-11-24T06:17:51.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.171+0000 7f658a7fc640 1 -- 172.21.15.5:0/152681571 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48633 (secure 0 0 0) 0x7f6584005e60 con 0x7f657806aed0 2023-11-24T06:17:51.173 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:51.174 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-11-24T06:17:51.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.174+0000 7f659bd89640 1 -- 172.21.15.5:0/152681571 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f657806aed0 msgr2=0x7f657806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:51.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.174+0000 7f659bd89640 1 --2- 172.21.15.5:0/152681571 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f657806aed0 0x7f657806d390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6584001eb0 tx=0x7f6584000f70 comp rx=0 tx=0).stop 2023-11-24T06:17:51.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.174+0000 7f659bd89640 1 -- 172.21.15.5:0/152681571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65800039a0 msgr2=0x7f659415dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:51.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.174+0000 7f659bd89640 1 --2- 172.21.15.5:0/152681571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65800039a0 0x7f659415dbc0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f659002c230 tx=0x7f6590004730 comp rx=0 tx=0).stop 2023-11-24T06:17:51.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.174+0000 7f659bd89640 1 -- 172.21.15.5:0/152681571 shutdown_connections 2023-11-24T06:17:51.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.174+0000 7f659bd89640 1 --2- 172.21.15.5:0/152681571 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f657806aed0 0x7f657806d390 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.174+0000 7f659bd89640 1 --2- 172.21.15.5:0/152681571 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f65940cccd0 0x7f6594150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.174+0000 7f659bd89640 1 --2- 172.21.15.5:0/152681571 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f659415e760 0x7f659415ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.174+0000 7f659bd89640 1 --2- 172.21.15.5:0/152681571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65800039a0 0x7f659415dbc0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.174+0000 7f659bd89640 1 -- 172.21.15.5:0/152681571 >> 172.21.15.5:0/152681571 conn(0x7f65940b0d80 msgr2=0x7f6594071050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:51.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.174+0000 7f659bd89640 1 -- 172.21.15.5:0/152681571 shutdown_connections 2023-11-24T06:17:51.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.174+0000 7f659bd89640 1 -- 172.21.15.5:0/152681571 wait complete. 2023-11-24T06:17:51.194 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-11-24T06:17:50.124916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":241452,"kb_used_data":4472,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733268,"statfs":{"total":773094113280,"available":772846866432,"internally_reserved":0,"allocated":4579328,"data_stored":2538553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33927,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001779"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":175,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391587+0000","last_change":"2023-11-24T06:15:42.021608+0000","last_active":"2023-11-24T06:17:18.391587+0000","last_peered":"2023-11-24T06:17:18.391587+0000","last_clean":"2023-11-24T06:17:18.391587+0000","last_became_active":"2023-11-24T06:15:42.018847+0000","last_became_peered":"2023-11-24T06:15:42.018847+0000","last_unstale":"2023-11-24T06:17:18.391587+0000","last_undegraded":"2023-11-24T06:17:18.391587+0000","last_fullsized":"2023-11-24T06:17:18.391587+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_clean_scrub_stamp":"2023-11-24T06:15:40.694368+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:41.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509467,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":987136,"data_stored":744396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2370000000000001,"15min":1.2509999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.663,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.238,"5min":1.2509999999999999,"15min":1.2529999999999999},"min":{"1min":0.70899999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7130000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.004}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.2250000000000001,"15min":1.2330000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.3700000000000001,"15min":1.3879999999999999},"min":{"1min":0.85399999999999998,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6830000000000001,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":1.226}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.4650000000000001,"15min":1.496},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.5980000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4930000000000001,"15min":1.51},"min":{"1min":0.86799999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.8280000000000001,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.5229999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.444,"15min":1.464},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.923,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.6519999999999999,"15min":1.6819999999999999},"min":{"1min":0.92200000000000004,"5min":0.879,"15min":0.879},"max":{"1min":1.857,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":1.3740000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.774,"15min":1.8180000000000001},"min":{"1min":1.081,"5min":1.016,"15min":1.016},"max":{"1min":2.141,"5min":5.7560000000000002,"15min":5.7560000000000002},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.581,"5min":1.669,"15min":1.6839999999999999},"min":{"1min":1.26,"5min":1.018,"15min":1.018},"max":{"1min":2.0910000000000002,"5min":2.681,"15min":2.681},"last":1.6850000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":2.0299999999999998,"15min":2.0880000000000001},"min":{"1min":1.0760000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.0960000000000001,"5min":6.6260000000000003,"15min":6.6260000000000003},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.663,"5min":2.0750000000000002,"15min":2.1440000000000001},"min":{"1min":1.155,"5min":1.155,"15min":1.155},"max":{"1min":2.0459999999999998,"5min":6.4169999999999998,"15min":6.4169999999999998},"last":1.155}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.7749999999999999,"5min":1.972,"15min":2.0049999999999999},"min":{"1min":1.214,"5min":1.214,"15min":1.214},"max":{"1min":2.198,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.633,"5min":1.853,"15min":1.889},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":2.173,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":1.587}]}]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":160123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.5449999999999999,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.0980000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.569,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.569}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.2370000000000001,"15min":1.246},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.77,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.266,"15min":1.2769999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.722,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.0940000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.246,"15min":1.2509999999999999},"min":{"1min":0.75700000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.994,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.4039999999999999,"15min":1.4199999999999999},"min":{"1min":0.84899999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.8029999999999999,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.4059999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.337,"15min":1.349},"min":{"1min":0.84099999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.796,"5min":2.331,"15min":2.331},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.53,"15min":1.55},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":1.3149999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.585,"15min":1.615},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.901,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.5229999999999999,"15min":1.5489999999999999},"min":{"1min":0.88600000000000001,"5min":0.873,"15min":0.873},"max":{"1min":1.9339999999999999,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.7450000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.6759999999999999,"15min":1.6930000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.464},{"interface":"front","average":{"1min":1.627,"5min":1.738,"15min":1.756},"min":{"1min":1.141,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":2.016,"5min":2.802,"15min":2.802},"last":1.6919999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.748,"15min":1.774},"min":{"1min":1.1399999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.048,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":1.851},{"interface":"front","average":{"1min":1.679,"5min":1.7549999999999999,"15min":1.7669999999999999},"min":{"1min":1.087,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.651}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":166022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.1679999999999999,"15min":1.204},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.482,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.72599999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6040000000000001,"5min":1.875,"15min":1.875},"last":0.98999999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.169,"15min":1.175},"min":{"1min":0.79300000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.609,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.802},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.282,"15min":1.3200000000000001},"min":{"1min":0.69499999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.526,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.24}]},{"osd":2,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.147,"15min":1.167},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.343,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.4690000000000001,"15min":1.5149999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.6399999999999999,"5min":2.633,"15min":2.633},"last":1.74}]},{"osd":3,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.3660000000000001,"15min":1.3939999999999999},"min":{"1min":0.85499999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5249999999999999,"5min":2.323,"15min":2.323},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.544,"15min":1.5820000000000001},"min":{"1min":0.90500000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.694,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":1.3109999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.5620000000000001,"15min":1.5900000000000001},"min":{"1min":0.64500000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7569999999999999,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":2.1219999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.49,"15min":1.5189999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.8899999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.1319999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.635,"15min":1.663},"min":{"1min":0.83499999999999996,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.845,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":1.401},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.579,"15min":1.627},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6890000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.6899999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.738,"15min":1.7709999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.0230000000000001,"5min":2.6880000000000002,"15min":2.6880000000000002},"last":1.502},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.6970000000000001,"15min":1.74},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.994,"5min":2.448,"15min":2.448},"last":1.7689999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0489999999999999,"15min":1.0369999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1220000000000001,"15min":1.1200000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.28}]},{"osd":1,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2529999999999999,"15min":1.2689999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.861,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.256,"15min":1.254},"min":{"1min":0.81100000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8200000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.6000000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.71899999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.55,"5min":1.677,"15min":1.677},"last":1.502},{"interface":"front","average":{"1min":1.359,"5min":1.419,"15min":1.429},"min":{"1min":0.72099999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.8520000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.3320000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2969999999999999,"15min":1.3029999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.655,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.5169999999999999,"15min":1.5269999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9450000000000001,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":1.45}]},{"osd":4,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.556,"15min":1.5580000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.95,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.8},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.82799999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.1899999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":1.7729999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.575,"15min":1.571},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.298,"5min":2.6480000000000001,"15min":2.6480000000000001},"last":2.109},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.6060000000000001,"15min":1.6220000000000001},"min":{"1min":1.028,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.887,"5min":3.1560000000000001,"15min":3.1560000000000001},"last":1.387}]},{"osd":7,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.655,"15min":1.657},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.23,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.6599999999999999,"5min":1.782,"15min":1.802},"min":{"1min":0.80100000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.1760000000000002,"5min":4.1779999999999999,"15min":4.1779999999999999},"last":1.728}]}]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":450560,"data_stored":197451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.93400000000000005,"15min":0.95099999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2210000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.03,"15min":1.044},"min":{"1min":0.48299999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.431,"5min":1.518,"15min":1.518},"last":1.1930000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.236,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.083,"15min":1.1259999999999999},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0980000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.3360000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.151,"15min":1.1870000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.603,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.405,"5min":1.875,"15min":1.875},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2310000000000001,"15min":1.2509999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.4770000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.022},{"interface":"front","average":{"1min":1.202,"5min":1.4370000000000001,"15min":1.476},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.776,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.5249999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.3480000000000001,"15min":1.375},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5660000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.137},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.5189999999999999,"15min":1.552},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.865,"5min":3.0089999999999999,"15min":3.0089999999999999},"last":1.5900000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.4319999999999999,"15min":1.4490000000000001},"min":{"1min":0.96699999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.8360000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.5,"15min":1.5229999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.944,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.3720000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.532,"15min":1.5589999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.9159999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.5780000000000001,"15min":1.609},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.869,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.546}]}]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31608,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.96099999999999997,"15min":0.94899999999999995},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.2709999999999999,"5min":5.2709999999999999,"15min":5.2709999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.157,"5min":1.079,"15min":1.0660000000000001},"min":{"1min":0.47499999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.5650000000000004,"5min":5.5650000000000004,"15min":5.5650000000000004},"last":1.2110000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1,"15min":0.99199999999999999},"min":{"1min":0.45700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":5.7140000000000004,"5min":5.7140000000000004,"15min":5.7140000000000004},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.0529999999999999,"15min":1.079},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.444,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":1.3759999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.131,"15min":1.1679999999999999},"min":{"1min":0.59599999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.403,"5min":1.95,"15min":1.95},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1839999999999999,"15min":1.1799999999999999},"min":{"1min":0.57699999999999996,"5min":0.504,"15min":0.504},"max":{"1min":5.4820000000000002,"5min":5.4820000000000002,"15min":5.4820000000000002},"last":1.466}]},{"osd":4,"last update":"Fri Nov 24 06:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.2789999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":4.1280000000000001,"5min":4.1280000000000001,"15min":4.1280000000000001},"last":1.121},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.4319999999999999,"15min":1.46},"min":{"1min":0.83099999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.7649999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.631}]},{"osd":6,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.427,"15min":1.4370000000000001},"min":{"1min":0.76700000000000002,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.6440000000000001,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":1.8120000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.357,"15min":1.383},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.2120000000000002,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.4379999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.681}]}]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":421888,"data_stored":166151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.89500000000000002,"15min":0.89100000000000001},"min":{"1min":0.66600000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.393,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.83599999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.91100000000000003,"15min":0.90100000000000002},"min":{"1min":0.72399999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3340000000000001,"5min":1.919,"15min":1.919},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.121,"15min":1.1120000000000001},"min":{"1min":0.71599999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6719999999999999,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.6719999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.0940000000000001,"15min":1.077},"min":{"1min":0.56399999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5940000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.579},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.004,"15min":0.998},"min":{"1min":0.54900000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.746,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":1.222}]},{"osd":4,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.3480000000000001,"15min":1.3340000000000001},"min":{"1min":1.2490000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7270000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.484},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3009999999999999,"15min":1.3049999999999999},"min":{"1min":0.90100000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.6579999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.542}]},{"osd":5,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.304,"15min":1.2909999999999999},"min":{"1min":0.88200000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.982,"5min":2.085,"15min":2.085},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3959999999999999,"15min":1.397},"min":{"1min":0.94499999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.6279999999999999,"5min":2.355,"15min":2.355},"last":1.2729999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.421,"15min":1.4159999999999999},"min":{"1min":1.046,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.944,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.4199999999999999,"15min":1.4119999999999999},"min":{"1min":0.96999999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8799999999999999,"5min":2.298,"15min":2.298},"last":1.3100000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.544,"15min":1.5389999999999999},"min":{"1min":1.218,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.0299999999999998,"5min":2.492,"15min":2.492},"last":1.383},{"interface":"front","average":{"1min":1.53,"5min":1.482,"15min":1.474},"min":{"1min":0.91400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.0830000000000002,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.7230000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1024000,"data_stored":761236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.89300000000000002,"15min":0.91900000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.127,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.95499999999999996,"15min":0.97499999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5600000000000001,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.4850000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.91100000000000003,"15min":0.92200000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3680000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.048,"15min":1.0549999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.6699999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":1.167}]},{"osd":3,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.996,"15min":1.01},"min":{"1min":0.495,"5min":0.434,"15min":0.434},"max":{"1min":1.444,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.999,"5min":1.0980000000000001,"15min":1.1140000000000001},"min":{"1min":0.59099999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.522,"5min":2.653,"15min":2.653},"last":1.266}]},{"osd":4,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3320000000000001,"15min":1.347},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.8260000000000001,"5min":2.254,"15min":2.254},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3740000000000001,"15min":1.3939999999999999},"min":{"1min":0.84499999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8620000000000001,"5min":1.996,"15min":1.996},"last":1.421}]},{"osd":5,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.4259999999999999,"15min":1.4399999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.956,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.744},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.4410000000000001,"15min":1.4690000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.919,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.623}]},{"osd":6,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.5029999999999999,"15min":1.518},"min":{"1min":0.90200000000000002,"5min":0.746,"15min":0.746},"max":{"1min":2.0939999999999999,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.494,"15min":1.52},"min":{"1min":0.78100000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.891,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.8680000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.5429999999999999,"15min":1.5509999999999999},"min":{"1min":0.89200000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.228},{"interface":"front","average":{"1min":1.359,"5min":1.4990000000000001,"15min":1.522},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.925,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.7709999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:17:51.195 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T06:17:51.195 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:17:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.418+0000 7f45e65a0640 1 Processor -- start 2023-11-24T06:17:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.418+0000 7f45e65a0640 1 -- start start 2023-11-24T06:17:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.420+0000 7f45e65a0640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f45e00cf050 0x7f45e00cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.420+0000 7f45e65a0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f45e00cd7b0 0x7f45e00cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.420+0000 7f45e65a0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e015fa70 con 0x7f45e013dfd0 2023-11-24T06:17:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.420+0000 7f45e65a0640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e00cfa40 con 0x7f45e00cd7b0 2023-11-24T06:17:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.420+0000 7f45e65a0640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e015c580 con 0x7f45e00cf050 2023-11-24T06:17:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.420+0000 7f45e559e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f45e013dfd0 0x7f45e015be70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43696/0 (socket says 172.21.15.5:43696) 2023-11-24T06:17:51.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.420+0000 7f45e559e640 1 -- 172.21.15.5:0/1026088289 learned_addr learned my addr 172.21.15.5:0/1026088289 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:51.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.420+0000 7f45e5d9f640 1 --2- 172.21.15.5:0/1026088289 >> v2:172.21.15.5:3300/0 conn(0x7f45e00cd7b0 0x7f45e00cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.420+0000 7f45e4d9d640 1 --2- 172.21.15.5:0/1026088289 >> v2:172.21.15.6:3300/0 conn(0x7f45e00cf050 0x7f45e00cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.421+0000 7f45d7fff640 1 -- 172.21.15.5:0/1026088289 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813038223 0 0) 0x7f45e015fa70 con 0x7f45e013dfd0 2023-11-24T06:17:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.421+0000 7f45e5d9f640 1 -- 172.21.15.5:0/1026088289 >> v2:172.21.15.6:3300/0 conn(0x7f45e00cf050 msgr2=0x7f45e00cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.421+0000 7f45e5d9f640 1 --2- 172.21.15.5:0/1026088289 >> v2:172.21.15.6:3300/0 conn(0x7f45e00cf050 0x7f45e00cf500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.421+0000 7f45e5d9f640 1 -- 172.21.15.5:0/1026088289 >> v1:172.21.15.5:6789/0 conn(0x7f45e013dfd0 legacy=0x7f45e015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.421+0000 7f45e5d9f640 1 -- 172.21.15.5:0/1026088289 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e0140900 con 0x7f45e00cd7b0 2023-11-24T06:17:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.422+0000 7f45e5d9f640 1 --2- 172.21.15.5:0/1026088289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e00cd7b0 0x7f45e00cdbb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f45e006e070 tx=0x7f45d802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48053e9d80ccc93 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.422+0000 7f45d7fff640 1 -- 172.21.15.5:0/1026088289 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d802cc70 con 0x7f45e00cd7b0 2023-11-24T06:17:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.422+0000 7f45d7fff640 1 -- 172.21.15.5:0/1026088289 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d802cdd0 con 0x7f45e00cd7b0 2023-11-24T06:17:51.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.422+0000 7f45d7fff640 1 -- 172.21.15.5:0/1026088289 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8034660 con 0x7f45e00cd7b0 2023-11-24T06:17:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.422+0000 7f45e65a0640 1 -- 172.21.15.5:0/1026088289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e00cd7b0 msgr2=0x7f45e00cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.422+0000 7f45e65a0640 1 --2- 172.21.15.5:0/1026088289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e00cd7b0 0x7f45e00cdbb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f45e006e070 tx=0x7f45d802bec0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.423+0000 7f45e65a0640 1 -- 172.21.15.5:0/1026088289 shutdown_connections 2023-11-24T06:17:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.423+0000 7f45e65a0640 1 --2- 172.21.15.5:0/1026088289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e00cd7b0 0x7f45e00cdbb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.423+0000 7f45e65a0640 1 --2- 172.21.15.5:0/1026088289 >> v2:172.21.15.6:3300/0 conn(0x7f45e00cf050 0x7f45e00cf500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.423+0000 7f45e65a0640 1 -- 172.21.15.5:0/1026088289 >> 172.21.15.5:0/1026088289 conn(0x7f45e006a860 msgr2=0x7f45e006ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.423+0000 7f45e65a0640 1 -- 172.21.15.5:0/1026088289 shutdown_connections 2023-11-24T06:17:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.423+0000 7f45e65a0640 1 -- 172.21.15.5:0/1026088289 wait complete. 2023-11-24T06:17:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.424+0000 7f45e65a0640 1 Processor -- start 2023-11-24T06:17:51.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.425+0000 7f45e65a0640 1 -- start start 2023-11-24T06:17:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.425+0000 7f45e65a0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e00cd7b0 0x7f45e00b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e65a0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45e00cf050 0x7f45e013b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e65a0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45e013dfd0 0x7f45e00b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e65a0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e015b730 con 0x7f45e00cd7b0 2023-11-24T06:17:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e65a0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e013bbd0 con 0x7f45e013dfd0 2023-11-24T06:17:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e65a0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00b1780 con 0x7f45e00cf050 2023-11-24T06:17:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e559e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e00cd7b0 0x7f45e00b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e559e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e00cd7b0 0x7f45e00b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50514/0 (socket says 172.21.15.5:50514) 2023-11-24T06:17:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e559e640 1 -- 172.21.15.5:0/3124137836 learned_addr learned my addr 172.21.15.5:0/3124137836 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e5d9f640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45e013dfd0 0x7f45e00b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e4d9d640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45e00cf050 0x7f45e013b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e559e640 1 -- 172.21.15.5:0/3124137836 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45e00cf050 msgr2=0x7f45e013b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e559e640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45e00cf050 0x7f45e013b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e559e640 1 -- 172.21.15.5:0/3124137836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45e013dfd0 msgr2=0x7f45e00b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e559e640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45e013dfd0 0x7f45e00b19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e559e640 1 -- 172.21.15.5:0/3124137836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d8009a70 con 0x7f45e00cd7b0 2023-11-24T06:17:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.426+0000 7f45e4d9d640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45e00cf050 0x7f45e013b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.427+0000 7f45e559e640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e00cd7b0 0x7f45e00b0820 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00c430 tx=0x7f45dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.427+0000 7f45d5ffb640 1 -- 172.21.15.5:0/3124137836 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0074e0 con 0x7f45e00cd7b0 2023-11-24T06:17:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.427+0000 7f45d5ffb640 1 -- 172.21.15.5:0/3124137836 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45dc007660 con 0x7f45e00cd7b0 2023-11-24T06:17:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.427+0000 7f45e65a0640 1 -- 172.21.15.5:0/3124137836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e00b0ee0 con 0x7f45e00cd7b0 2023-11-24T06:17:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.427+0000 7f45d5ffb640 1 -- 172.21.15.5:0/3124137836 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45dc004a20 con 0x7f45e00cd7b0 2023-11-24T06:17:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.427+0000 7f45e65a0640 1 -- 172.21.15.5:0/3124137836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e00b14a0 con 0x7f45e00cd7b0 2023-11-24T06:17:51.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.428+0000 7f45e65a0640 1 -- 172.21.15.5:0/3124137836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e004e840 con 0x7f45e00cd7b0 2023-11-24T06:17:51.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.429+0000 7f45d5ffb640 1 -- 172.21.15.5:0/3124137836 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f45dc004b80 con 0x7f45e00cd7b0 2023-11-24T06:17:51.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.431+0000 7f45d5ffb640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f45c406aed0 0x7f45c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:51.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.431+0000 7f45d5ffb640 1 -- 172.21.15.5:0/3124137836 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f45dc089220 con 0x7f45e00cd7b0 2023-11-24T06:17:51.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.431+0000 7f45e4d9d640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f45c406aed0 0x7f45c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:51.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.432+0000 7f45e4d9d640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f45c406aed0 0x7f45c406d390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f45e01589c0 tx=0x7f45d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:51.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.435+0000 7f45d5ffb640 1 -- 172.21.15.5:0/3124137836 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45dc054570 con 0x7f45e00cd7b0 2023-11-24T06:17:51.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.624+0000 7f45e65a0640 1 -- 172.21.15.5:0/3124137836 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f45e015f420 con 0x7f45c406aed0 2023-11-24T06:17:51.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.627+0000 7f45d5ffb640 1 -- 172.21.15.5:0/3124137836 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48633 (secure 0 0 0) 0x7f45d0009e10 con 0x7f45c406aed0 2023-11-24T06:17:51.629 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:51.630 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-11-24T06:17:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.629+0000 7f45e65a0640 1 -- 172.21.15.5:0/3124137836 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f45c406aed0 msgr2=0x7f45c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.629+0000 7f45e65a0640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f45c406aed0 0x7f45c406d390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f45e01589c0 tx=0x7f45d0008040 comp rx=0 tx=0).stop 2023-11-24T06:17:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.629+0000 7f45e65a0640 1 -- 172.21.15.5:0/3124137836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e00cd7b0 msgr2=0x7f45e00b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.629+0000 7f45e65a0640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e00cd7b0 0x7f45e00b0820 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00c430 tx=0x7f45dc00c8f0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.629+0000 7f45e65a0640 1 -- 172.21.15.5:0/3124137836 shutdown_connections 2023-11-24T06:17:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.629+0000 7f45e65a0640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f45c406aed0 0x7f45c406d390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.629+0000 7f45e65a0640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f45e00cf050 0x7f45e013b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.629+0000 7f45e65a0640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f45e013dfd0 0x7f45e00b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.629+0000 7f45e65a0640 1 --2- 172.21.15.5:0/3124137836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e00cd7b0 0x7f45e00b0820 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.629+0000 7f45e65a0640 1 -- 172.21.15.5:0/3124137836 >> 172.21.15.5:0/3124137836 conn(0x7f45e006a860 msgr2=0x7f45e013ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:51.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.629+0000 7f45e65a0640 1 -- 172.21.15.5:0/3124137836 shutdown_connections 2023-11-24T06:17:51.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.629+0000 7f45e65a0640 1 -- 172.21.15.5:0/3124137836 wait complete. 2023-11-24T06:17:51.651 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-11-24T06:17:50.124916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":241452,"kb_used_data":4472,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733268,"statfs":{"total":773094113280,"available":772846866432,"internally_reserved":0,"allocated":4579328,"data_stored":2538553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-33927,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-66,"num_write_kb":-56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001779"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":175,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T06:17:18.391587+0000","last_change":"2023-11-24T06:15:42.021608+0000","last_active":"2023-11-24T06:17:18.391587+0000","last_peered":"2023-11-24T06:17:18.391587+0000","last_clean":"2023-11-24T06:17:18.391587+0000","last_became_active":"2023-11-24T06:15:42.018847+0000","last_became_peered":"2023-11-24T06:15:42.018847+0000","last_unstale":"2023-11-24T06:17:18.391587+0000","last_undegraded":"2023-11-24T06:17:18.391587+0000","last_fullsized":"2023-11-24T06:17:18.391587+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_clean_scrub_stamp":"2023-11-24T06:15:40.694368+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:41.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509467,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":987136,"data_stored":744396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2370000000000001,"15min":1.2509999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.663,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.238,"5min":1.2509999999999999,"15min":1.2529999999999999},"min":{"1min":0.70899999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7130000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.004}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.2250000000000001,"15min":1.2330000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.3700000000000001,"15min":1.3879999999999999},"min":{"1min":0.85399999999999998,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6830000000000001,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":1.226}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.4650000000000001,"15min":1.496},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.5980000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4930000000000001,"15min":1.51},"min":{"1min":0.86799999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.8280000000000001,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.5229999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.444,"15min":1.464},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.923,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.476,"5min":1.6519999999999999,"15min":1.6819999999999999},"min":{"1min":0.92200000000000004,"5min":0.879,"15min":0.879},"max":{"1min":1.857,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":1.3740000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.774,"15min":1.8180000000000001},"min":{"1min":1.081,"5min":1.016,"15min":1.016},"max":{"1min":2.141,"5min":5.7560000000000002,"15min":5.7560000000000002},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.581,"5min":1.669,"15min":1.6839999999999999},"min":{"1min":1.26,"5min":1.018,"15min":1.018},"max":{"1min":2.0910000000000002,"5min":2.681,"15min":2.681},"last":1.6850000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":2.0299999999999998,"15min":2.0880000000000001},"min":{"1min":1.0760000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.0960000000000001,"5min":6.6260000000000003,"15min":6.6260000000000003},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.663,"5min":2.0750000000000002,"15min":2.1440000000000001},"min":{"1min":1.155,"5min":1.155,"15min":1.155},"max":{"1min":2.0459999999999998,"5min":6.4169999999999998,"15min":6.4169999999999998},"last":1.155}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.7749999999999999,"5min":1.972,"15min":2.0049999999999999},"min":{"1min":1.214,"5min":1.214,"15min":1.214},"max":{"1min":2.198,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":1.6319999999999999},{"interface":"front","average":{"1min":1.633,"5min":1.853,"15min":1.889},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":2.173,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":1.587}]}]},{"osd":4,"up_from":15,"seq":64424509468,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":409600,"data_stored":160123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.5449999999999999,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.0980000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.569,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.569}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.2370000000000001,"15min":1.246},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.77,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.266,"15min":1.2769999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.722,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.0940000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.246,"15min":1.2509999999999999},"min":{"1min":0.75700000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.994,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.4039999999999999,"15min":1.4199999999999999},"min":{"1min":0.84899999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.8029999999999999,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.4059999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.337,"15min":1.349},"min":{"1min":0.84099999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.796,"5min":2.331,"15min":2.331},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.53,"15min":1.55},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":1.3149999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.585,"15min":1.615},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.901,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.5229999999999999,"15min":1.5489999999999999},"min":{"1min":0.88600000000000001,"5min":0.873,"15min":0.873},"max":{"1min":1.9339999999999999,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.7450000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.6759999999999999,"15min":1.6930000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.464},{"interface":"front","average":{"1min":1.627,"5min":1.738,"15min":1.756},"min":{"1min":1.141,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":2.016,"5min":2.802,"15min":2.802},"last":1.6919999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.748,"15min":1.774},"min":{"1min":1.1399999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.048,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":1.851},{"interface":"front","average":{"1min":1.679,"5min":1.7549999999999999,"15min":1.7669999999999999},"min":{"1min":1.087,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.651}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27428,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":417792,"data_stored":166022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.1679999999999999,"15min":1.204},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.482,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.72599999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6040000000000001,"5min":1.875,"15min":1.875},"last":0.98999999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.169,"15min":1.175},"min":{"1min":0.79300000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.609,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.802},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.282,"15min":1.3200000000000001},"min":{"1min":0.69499999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.526,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.24}]},{"osd":2,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.147,"15min":1.167},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.343,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.4690000000000001,"15min":1.5149999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.6399999999999999,"5min":2.633,"15min":2.633},"last":1.74}]},{"osd":3,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.3660000000000001,"15min":1.3939999999999999},"min":{"1min":0.85499999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5249999999999999,"5min":2.323,"15min":2.323},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.544,"15min":1.5820000000000001},"min":{"1min":0.90500000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.694,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":1.3109999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.5620000000000001,"15min":1.5900000000000001},"min":{"1min":0.64500000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7569999999999999,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":2.1219999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.49,"15min":1.5189999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.8899999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.1319999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.635,"15min":1.663},"min":{"1min":0.83499999999999996,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.845,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":1.401},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.579,"15min":1.627},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6890000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.6899999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.738,"15min":1.7709999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.0230000000000001,"5min":2.6880000000000002,"15min":2.6880000000000002},"last":1.502},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.6970000000000001,"15min":1.74},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.994,"5min":2.448,"15min":2.448},"last":1.7689999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0489999999999999,"15min":1.0369999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.131,"5min":1.1220000000000001,"15min":1.1200000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.28}]},{"osd":1,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2529999999999999,"15min":1.2689999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.861,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.256,"15min":1.254},"min":{"1min":0.81100000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8200000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.6000000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.71899999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.55,"5min":1.677,"15min":1.677},"last":1.502},{"interface":"front","average":{"1min":1.359,"5min":1.419,"15min":1.429},"min":{"1min":0.72099999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.8520000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.3320000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2969999999999999,"15min":1.3029999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.655,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.6679999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.5169999999999999,"15min":1.5269999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9450000000000001,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":1.45}]},{"osd":4,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.556,"15min":1.5580000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.95,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.8},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.82799999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.1899999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":1.7729999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.575,"15min":1.571},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.298,"5min":2.6480000000000001,"15min":2.6480000000000001},"last":2.109},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.6060000000000001,"15min":1.6220000000000001},"min":{"1min":1.028,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.887,"5min":3.1560000000000001,"15min":3.1560000000000001},"last":1.387}]},{"osd":7,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.655,"15min":1.657},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.23,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.6599999999999999,"5min":1.782,"15min":1.802},"min":{"1min":0.80100000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.1760000000000002,"5min":4.1779999999999999,"15min":4.1779999999999999},"last":1.728}]}]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31620,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340220,"statfs":{"total":96636764160,"available":96604385280,"internally_reserved":0,"allocated":450560,"data_stored":197451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.93400000000000005,"15min":0.95099999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2210000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.2509999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.03,"15min":1.044},"min":{"1min":0.48299999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.431,"5min":1.518,"15min":1.518},"last":1.1930000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.236,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.083,"15min":1.1259999999999999},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0980000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.3360000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.151,"15min":1.1870000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.603,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.405,"5min":1.875,"15min":1.875},"last":0.77400000000000002}]},{"osd":4,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2310000000000001,"15min":1.2509999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.4770000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.022},{"interface":"front","average":{"1min":1.202,"5min":1.4370000000000001,"15min":1.476},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.776,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.5249999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.3480000000000001,"15min":1.375},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5660000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.137},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.5189999999999999,"15min":1.552},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.865,"5min":3.0089999999999999,"15min":3.0089999999999999},"last":1.5900000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.4319999999999999,"15min":1.4490000000000001},"min":{"1min":0.96699999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.8360000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.361,"5min":1.5,"15min":1.5229999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.944,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.3720000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.532,"15min":1.5589999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.9159999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.5780000000000001,"15min":1.609},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.869,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.546}]}]},{"osd":2,"up_from":14,"seq":60129542173,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31608,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.96099999999999997,"15min":0.94899999999999995},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.2709999999999999,"5min":5.2709999999999999,"15min":5.2709999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.157,"5min":1.079,"15min":1.0660000000000001},"min":{"1min":0.47499999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.5650000000000004,"5min":5.5650000000000004,"15min":5.5650000000000004},"last":1.2110000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1,"15min":0.99199999999999999},"min":{"1min":0.45700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":5.7140000000000004,"5min":5.7140000000000004,"15min":5.7140000000000004},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.0529999999999999,"15min":1.079},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.444,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":1.3759999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.131,"15min":1.1679999999999999},"min":{"1min":0.59599999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.403,"5min":1.95,"15min":1.95},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1839999999999999,"15min":1.1799999999999999},"min":{"1min":0.57699999999999996,"5min":0.504,"15min":0.504},"max":{"1min":5.4820000000000002,"5min":5.4820000000000002,"15min":5.4820000000000002},"last":1.466}]},{"osd":4,"last update":"Fri Nov 24 06:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.6919999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":1.2789999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":4.1280000000000001,"5min":4.1280000000000001,"15min":4.1280000000000001},"last":1.121},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.4319999999999999,"15min":1.46},"min":{"1min":0.83099999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.7649999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.631}]},{"osd":6,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.427,"15min":1.4370000000000001},"min":{"1min":0.76700000000000002,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.6440000000000001,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":1.8120000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.357,"15min":1.383},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.2120000000000002,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.4379999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.7589999999999999},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"min":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.681}]}]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":421888,"data_stored":166151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.89500000000000002,"15min":0.89100000000000001},"min":{"1min":0.66600000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.393,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.83599999999999997}]},{"osd":2,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.91100000000000003,"15min":0.90100000000000002},"min":{"1min":0.72399999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3340000000000001,"5min":1.919,"15min":1.919},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.121,"15min":1.1120000000000001},"min":{"1min":0.71599999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6719999999999999,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.6719999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.0940000000000001,"15min":1.077},"min":{"1min":0.56399999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5940000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.579},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.004,"15min":0.998},"min":{"1min":0.54900000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.746,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":1.222}]},{"osd":4,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.3480000000000001,"15min":1.3340000000000001},"min":{"1min":1.2490000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7270000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.484},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3009999999999999,"15min":1.3049999999999999},"min":{"1min":0.90100000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.6579999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.542}]},{"osd":5,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.304,"15min":1.2909999999999999},"min":{"1min":0.88200000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.982,"5min":2.085,"15min":2.085},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3959999999999999,"15min":1.397},"min":{"1min":0.94499999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.6279999999999999,"5min":2.355,"15min":2.355},"last":1.2729999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.421,"15min":1.4159999999999999},"min":{"1min":1.046,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.944,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.4199999999999999,"15min":1.4119999999999999},"min":{"1min":0.96999999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8799999999999999,"5min":2.298,"15min":2.298},"last":1.3100000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.544,"15min":1.5389999999999999},"min":{"1min":1.218,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.0299999999999998,"5min":2.492,"15min":2.492},"last":1.383},{"interface":"front","average":{"1min":1.53,"5min":1.482,"15min":1.474},"min":{"1min":0.91400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.0830000000000002,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.7230000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1024000,"data_stored":761236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.89300000000000002,"15min":0.91900000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.127,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.95499999999999996,"15min":0.97499999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5600000000000001,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.4850000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.91100000000000003,"15min":0.92200000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3680000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.048,"15min":1.0549999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.6699999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":1.167}]},{"osd":3,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.996,"15min":1.01},"min":{"1min":0.495,"5min":0.434,"15min":0.434},"max":{"1min":1.444,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.999,"5min":1.0980000000000001,"15min":1.1140000000000001},"min":{"1min":0.59099999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.522,"5min":2.653,"15min":2.653},"last":1.266}]},{"osd":4,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3320000000000001,"15min":1.347},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.8260000000000001,"5min":2.254,"15min":2.254},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3740000000000001,"15min":1.3939999999999999},"min":{"1min":0.84499999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8620000000000001,"5min":1.996,"15min":1.996},"last":1.421}]},{"osd":5,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.4259999999999999,"15min":1.4399999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.956,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.744},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.4410000000000001,"15min":1.4690000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.919,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.623}]},{"osd":6,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.5029999999999999,"15min":1.518},"min":{"1min":0.90200000000000002,"5min":0.746,"15min":0.746},"max":{"1min":2.0939999999999999,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.494,"15min":1.52},"min":{"1min":0.78100000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.891,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.8680000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.5429999999999999,"15min":1.5509999999999999},"min":{"1min":0.89200000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.228},{"interface":"front","average":{"1min":1.359,"5min":1.4990000000000001,"15min":1.522},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.925,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.7709999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:17:51.652 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T06:17:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.875+0000 7ff7adf33640 1 Processor -- start 2023-11-24T06:17:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.875+0000 7ff7adf33640 1 -- start start 2023-11-24T06:17:51.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.877+0000 7ff7adf33640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7ff7a80ad720 0x7ff7a80b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:51.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.877+0000 7ff7adf33640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a815c360 con 0x7ff7a806d520 2023-11-24T06:17:51.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.877+0000 7ff7adf33640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a815ba30 con 0x7ff7a80abff0 2023-11-24T06:17:51.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.877+0000 7ff7adf33640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a80b41a0 con 0x7ff7a80ad720 2023-11-24T06:17:51.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.877+0000 7ff7a77fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff7a806d520 0x7ff7a806d900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:43708/0 (socket says 172.21.15.5:43708) 2023-11-24T06:17:51.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.877+0000 7ff7a77fe640 1 -- 172.21.15.5:0/3990295749 learned_addr learned my addr 172.21.15.5:0/3990295749 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:51.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.878+0000 7ff79effd640 1 --2- 172.21.15.5:0/3990295749 >> v2:172.21.15.6:3300/0 conn(0x7ff7a80ad720 0x7ff7a80b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:51.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.878+0000 7ff7a6ffd640 1 -- 172.21.15.5:0/3990295749 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510344566 0 0) 0x7ff7a815c360 con 0x7ff7a806d520 2023-11-24T06:17:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.878+0000 7ff7a6ffd640 1 -- 172.21.15.5:0/3990295749 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff794003520 con 0x7ff7a806d520 2023-11-24T06:17:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.878+0000 7ff7a6ffd640 1 -- 172.21.15.5:0/3990295749 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3191314706 0 0) 0x7ff794003520 con 0x7ff7a806d520 2023-11-24T06:17:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.878+0000 7ff7a6ffd640 1 -- 172.21.15.5:0/3990295749 >> v2:172.21.15.6:3300/0 conn(0x7ff7a80ad720 msgr2=0x7ff7a80b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.878+0000 7ff7a6ffd640 1 --2- 172.21.15.5:0/3990295749 >> v2:172.21.15.6:3300/0 conn(0x7ff7a80ad720 0x7ff7a80b2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.878+0000 7ff7a6ffd640 1 -- 172.21.15.5:0/3990295749 >> v1:172.21.15.6:6789/0 conn(0x7ff7a80abff0 legacy=0x7ff7a80b25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.878+0000 7ff7a6ffd640 1 -- 172.21.15.5:0/3990295749 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a815bd10 con 0x7ff7a806d520 2023-11-24T06:17:51.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.878+0000 7ff7a6ffd640 1 -- 172.21.15.5:0/3990295749 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7ff7980019f0 con 0x7ff7a806d520 2023-11-24T06:17:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.878+0000 7ff7a6ffd640 1 -- 172.21.15.5:0/3990295749 >> v1:172.21.15.5:6789/0 conn(0x7ff7a806d520 legacy=0x7ff7a806d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff7a6ffd640 1 --2- 172.21.15.5:0/3990295749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7940039c0 0x7ff794003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff7a6ffd640 1 --2- 172.21.15.5:0/3990295749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7940047c0 0x7ff794004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff7a6ffd640 1 --2- 172.21.15.5:0/3990295749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff794005180 0x7ff79400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff7a6ffd640 1 -- 172.21.15.5:0/3990295749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79400c050 con 0x7ff7940047c0 2023-11-24T06:17:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff7a6ffd640 1 -- 172.21.15.5:0/3990295749 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79400c1c0 con 0x7ff794005180 2023-11-24T06:17:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff7a6ffd640 1 -- 172.21.15.5:0/3990295749 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79400c380 con 0x7ff7940039c0 2023-11-24T06:17:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff79effd640 1 --2- 172.21.15.5:0/3990295749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7940047c0 0x7ff794004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:51.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff7a77fe640 1 --2- 172.21.15.5:0/3990295749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7940039c0 0x7ff794003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:51.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff79effd640 1 -- 172.21.15.5:0/3990295749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7940039c0 msgr2=0x7ff794003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:51.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff79effd640 1 --2- 172.21.15.5:0/3990295749 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7940039c0 0x7ff794003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff79effd640 1 -- 172.21.15.5:0/3990295749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff794005180 msgr2=0x7ff79400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:51.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff7a7fff640 1 --2- 172.21.15.5:0/3990295749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff794005180 0x7ff79400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:51.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff79effd640 1 --2- 172.21.15.5:0/3990295749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff794005180 0x7ff79400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff79effd640 1 -- 172.21.15.5:0/3990295749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff794003520 con 0x7ff7940047c0 2023-11-24T06:17:51.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff7a7fff640 1 --2- 172.21.15.5:0/3990295749 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff794005180 0x7ff79400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:51.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.879+0000 7ff7a7fff640 1 -- 172.21.15.5:0/3990295749 reap_dead start 2023-11-24T06:17:51.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.881+0000 7ff79effd640 1 --2- 172.21.15.5:0/3990295749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7940047c0 0x7ff794004c40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff790009a90 tx=0x7ff7940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0391f580bb3f17 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:51.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.881+0000 7ff7a6ffd640 1 -- 172.21.15.5:0/3990295749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79002c680 con 0x7ff7940047c0 2023-11-24T06:17:51.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.881+0000 7ff7a6ffd640 1 -- 172.21.15.5:0/3990295749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79002c7e0 con 0x7ff7940047c0 2023-11-24T06:17:51.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.881+0000 7ff7a6ffd640 1 -- 172.21.15.5:0/3990295749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79002cb90 con 0x7ff7940047c0 2023-11-24T06:17:51.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.881+0000 7ff7adf33640 1 -- 172.21.15.5:0/3990295749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7940047c0 msgr2=0x7ff794004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:51.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.881+0000 7ff7adf33640 1 --2- 172.21.15.5:0/3990295749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7940047c0 0x7ff794004c40 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff790009a90 tx=0x7ff7940057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.882+0000 7ff7adf33640 1 -- 172.21.15.5:0/3990295749 shutdown_connections 2023-11-24T06:17:51.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.882+0000 7ff7adf33640 1 --2- 172.21.15.5:0/3990295749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7940047c0 0x7ff794004c40 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.882+0000 7ff7adf33640 1 -- 172.21.15.5:0/3990295749 >> 172.21.15.5:0/3990295749 conn(0x7ff7a80b0d80 msgr2=0x7ff7a80b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:51.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.882+0000 7ff7adf33640 1 -- 172.21.15.5:0/3990295749 shutdown_connections 2023-11-24T06:17:51.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.882+0000 7ff7adf33640 1 -- 172.21.15.5:0/3990295749 wait complete. 2023-11-24T06:17:51.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.884+0000 7ff7adf33640 1 Processor -- start 2023-11-24T06:17:51.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.884+0000 7ff7adf33640 1 -- start start 2023-11-24T06:17:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.885+0000 7ff7adf33640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7940047c0 0x7ff7a815a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.885+0000 7ff7adf33640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7a815a8c0 0x7ff7a815ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.885+0000 7ff7adf33640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7a80704b0 0x7ff7a8070930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.885+0000 7ff7adf33640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a80cc000 con 0x7ff7a815a8c0 2023-11-24T06:17:51.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.885+0000 7ff7adf33640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a81592e0 con 0x7ff7a80704b0 2023-11-24T06:17:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.885+0000 7ff7adf33640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a81595c0 con 0x7ff7940047c0 2023-11-24T06:17:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.885+0000 7ff79effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7a815a8c0 0x7ff7a815ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.885+0000 7ff79effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7a815a8c0 0x7ff7a815ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50530/0 (socket says 172.21.15.5:50530) 2023-11-24T06:17:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.885+0000 7ff79effd640 1 -- 172.21.15.5:0/4266902444 learned_addr learned my addr 172.21.15.5:0/4266902444 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.885+0000 7ff7a77fe640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7940047c0 0x7ff7a815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.885+0000 7ff79effd640 1 -- 172.21.15.5:0/4266902444 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7940047c0 msgr2=0x7ff7a815a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.885+0000 7ff79effd640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7940047c0 0x7ff7a815a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.885+0000 7ff7a7fff640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7a80704b0 0x7ff7a8070930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.885+0000 7ff79effd640 1 -- 172.21.15.5:0/4266902444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7a80704b0 msgr2=0x7ff7a8070930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.886+0000 7ff79effd640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7a80704b0 0x7ff7a8070930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.886+0000 7ff79effd640 1 -- 172.21.15.5:0/4266902444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a81598a0 con 0x7ff7a815a8c0 2023-11-24T06:17:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.886+0000 7ff7a7fff640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7a80704b0 0x7ff7a8070930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.886+0000 7ff79effd640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7a815a8c0 0x7ff7a815ad40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff790009bb0 tx=0x7ff790004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.886+0000 7ff7a4ff9640 1 -- 172.21.15.5:0/4266902444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff790034a90 con 0x7ff7a815a8c0 2023-11-24T06:17:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.886+0000 7ff7a4ff9640 1 -- 172.21.15.5:0/4266902444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff790034c10 con 0x7ff7a815a8c0 2023-11-24T06:17:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.886+0000 7ff7a4ff9640 1 -- 172.21.15.5:0/4266902444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79003e400 con 0x7ff7a815a8c0 2023-11-24T06:17:51.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.886+0000 7ff7adf33640 1 -- 172.21.15.5:0/4266902444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a8159b80 con 0x7ff7a815a8c0 2023-11-24T06:17:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.886+0000 7ff7adf33640 1 -- 172.21.15.5:0/4266902444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a815b280 con 0x7ff7a815a8c0 2023-11-24T06:17:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.888+0000 7ff7adf33640 1 -- 172.21.15.5:0/4266902444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a8074770 con 0x7ff7a815a8c0 2023-11-24T06:17:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.888+0000 7ff7a4ff9640 1 -- 172.21.15.5:0/4266902444 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ff79003e560 con 0x7ff7a815a8c0 2023-11-24T06:17:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.889+0000 7ff7a4ff9640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff78806aed0 0x7ff78806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.889+0000 7ff7a4ff9640 1 -- 172.21.15.5:0/4266902444 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff7900ab390 con 0x7ff7a815a8c0 2023-11-24T06:17:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.890+0000 7ff7a77fe640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff78806aed0 0x7ff78806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:51.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.890+0000 7ff7a77fe640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff78806aed0 0x7ff78806d390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff798001370 tx=0x7ff798009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:51.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:51.894+0000 7ff7a4ff9640 1 -- 172.21.15.5:0/4266902444 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff790076700 con 0x7ff7a815a8c0 2023-11-24T06:17:52.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:52.091+0000 7ff7adf33640 1 -- 172.21.15.5:0/4266902444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7a80cc000 con 0x7ff7a815a8c0 2023-11-24T06:17:52.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:52.093+0000 7ff7a4ff9640 1 -- 172.21.15.5:0/4266902444 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+12840 (secure 0 0 0) 0x7ff790079d20 con 0x7ff7a815a8c0 2023-11-24T06:17:52.095 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:52.095 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":38,"fsid":"8bd5ce8c-8c1e-424d-996c-660bfd1d7d5c","created":"2023-11-24T06:15:27.036335+0000","modified":"2023-11-24T06:17:47.994728+0000","last_up_change":"2023-11-24T06:15:39.689667+0000","last_in_change":"2023-11-24T06:15:30.647672+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T06:15:40.164863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"657f6d5f-230f-4b17-8906-54b4d9352abd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6817","nonce":1571096651}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6819","nonce":1571096651}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6823","nonce":1571096651}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1571096651},{"type":"v1","addr":"172.21.15.5:6821","nonce":1571096651}]},"public_addr":"172.21.15.5:6817/1571096651","cluster_addr":"172.21.15.5:6819/1571096651","heartbeat_back_addr":"172.21.15.5:6823/1571096651","heartbeat_front_addr":"172.21.15.5:6821/1571096651","state":["exists","up"]},{"osd":1,"uuid":"00ada985-d8e1-4b65-bbb6-e7a0d651c82c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6801","nonce":1318127149}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6803","nonce":1318127149}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6807","nonce":1318127149}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":1318127149},{"type":"v1","addr":"172.21.15.5:6805","nonce":1318127149}]},"public_addr":"172.21.15.5:6801/1318127149","cluster_addr":"172.21.15.5:6803/1318127149","heartbeat_back_addr":"172.21.15.5:6807/1318127149","heartbeat_front_addr":"172.21.15.5:6805/1318127149","state":["exists","up"]},{"osd":2,"uuid":"d0bd3e3f-00e3-49d4-a7e9-9c34c499b5bf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6825","nonce":1782469371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6827","nonce":1782469371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6831","nonce":1782469371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1782469371},{"type":"v1","addr":"172.21.15.5:6829","nonce":1782469371}]},"public_addr":"172.21.15.5:6825/1782469371","cluster_addr":"172.21.15.5:6827/1782469371","heartbeat_back_addr":"172.21.15.5:6831/1782469371","heartbeat_front_addr":"172.21.15.5:6829/1782469371","state":["exists","up"]},{"osd":3,"uuid":"388d7df5-5c6c-45db-832b-a306b901fe33","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6809","nonce":337602632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6811","nonce":337602632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6815","nonce":337602632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":337602632},{"type":"v1","addr":"172.21.15.5:6813","nonce":337602632}]},"public_addr":"172.21.15.5:6809/337602632","cluster_addr":"172.21.15.5:6811/337602632","heartbeat_back_addr":"172.21.15.5:6815/337602632","heartbeat_front_addr":"172.21.15.5:6813/337602632","state":["exists","up"]},{"osd":4,"uuid":"c12b3cb8-7a92-4dd8-9499-3ee17f6a2934","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6822","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6823","nonce":2056352444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6826","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6827","nonce":2056352444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6830","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6831","nonce":2056352444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6828","nonce":2056352444},{"type":"v1","addr":"172.21.15.6:6829","nonce":2056352444}]},"public_addr":"172.21.15.6:6823/2056352444","cluster_addr":"172.21.15.6:6827/2056352444","heartbeat_back_addr":"172.21.15.6:6831/2056352444","heartbeat_front_addr":"172.21.15.6:6829/2056352444","state":["exists","up"]},{"osd":5,"uuid":"a83ba2b6-0af5-4d7d-bbb1-27a867e1acb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6800","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6801","nonce":3179572934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6802","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6803","nonce":3179572934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6806","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6807","nonce":3179572934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6804","nonce":3179572934},{"type":"v1","addr":"172.21.15.6:6805","nonce":3179572934}]},"public_addr":"172.21.15.6:6801/3179572934","cluster_addr":"172.21.15.6:6803/3179572934","heartbeat_back_addr":"172.21.15.6:6807/3179572934","heartbeat_front_addr":"172.21.15.6:6805/3179572934","state":["exists","up"]},{"osd":6,"uuid":"f31a6b12-4ba8-494e-b00d-79c187dc5aad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6814","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6816","nonce":2361559103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6818","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6819","nonce":2361559103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6824","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6825","nonce":2361559103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6820","nonce":2361559103},{"type":"v1","addr":"172.21.15.6:6821","nonce":2361559103}]},"public_addr":"172.21.15.6:6816/2361559103","cluster_addr":"172.21.15.6:6819/2361559103","heartbeat_back_addr":"172.21.15.6:6825/2361559103","heartbeat_front_addr":"172.21.15.6:6821/2361559103","state":["exists","up"]},{"osd":7,"uuid":"f3ee2ac0-f1db-4dee-80cc-ce8998d7259e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6808","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6809","nonce":4073132861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6810","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6811","nonce":4073132861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6815","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6817","nonce":4073132861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6812","nonce":4073132861},{"type":"v1","addr":"172.21.15.6:6813","nonce":4073132861}]},"public_addr":"172.21.15.6:6809/4073132861","cluster_addr":"172.21.15.6:6811/4073132861","heartbeat_back_addr":"172.21.15.6:6817/4073132861","heartbeat_front_addr":"172.21.15.6:6813/4073132861","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.976480+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.212531+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.076177+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.194806+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.340114+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:35.964420+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.056915+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T06:15:36.319438+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6837/3690265514":"2023-11-25T06:17:44.949002+0000","172.21.15.5:6835/3690265514":"2023-11-25T06:17:44.949002+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T06:17:52.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:52.096+0000 7ff7adf33640 1 -- 172.21.15.5:0/4266902444 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff78806aed0 msgr2=0x7ff78806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:52.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:52.096+0000 7ff7adf33640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff78806aed0 0x7ff78806d390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff798001370 tx=0x7ff798009040 comp rx=0 tx=0).stop 2023-11-24T06:17:52.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:52.096+0000 7ff7adf33640 1 -- 172.21.15.5:0/4266902444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7a815a8c0 msgr2=0x7ff7a815ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:52.096+0000 7ff7adf33640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7a815a8c0 0x7ff7a815ad40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff790009bb0 tx=0x7ff790004730 comp rx=0 tx=0).stop 2023-11-24T06:17:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:52.096+0000 7ff7adf33640 1 -- 172.21.15.5:0/4266902444 shutdown_connections 2023-11-24T06:17:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:52.096+0000 7ff7adf33640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff78806aed0 0x7ff78806d390 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:52.096+0000 7ff7adf33640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7a80704b0 0x7ff7a8070930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:52.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:52.096+0000 7ff7adf33640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7a815a8c0 0x7ff7a815ad40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:52.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:52.096+0000 7ff7adf33640 1 --2- 172.21.15.5:0/4266902444 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7940047c0 0x7ff7a815a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:52.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:52.096+0000 7ff7adf33640 1 -- 172.21.15.5:0/4266902444 >> 172.21.15.5:0/4266902444 conn(0x7ff7a80b0d80 msgr2=0x7ff7a80b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:52.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:52.096+0000 7ff7adf33640 1 -- 172.21.15.5:0/4266902444 shutdown_connections 2023-11-24T06:17:52.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:52.096+0000 7ff7adf33640 1 -- 172.21.15.5:0/4266902444 wait complete. 2023-11-24T06:17:53.118 INFO:tasks.ceph:Scrubbing osd.0 2023-11-24T06:17:53.119 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:17:53.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.360+0000 7efcfbfff640 1 Processor -- start 2023-11-24T06:17:53.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.360+0000 7efcfbfff640 1 -- start start 2023-11-24T06:17:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.362+0000 7efcfbfff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7efcfc06d9e0 0x7efcfc165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.362+0000 7efcfbfff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcfc15f930 con 0x7efcfc15f270 2023-11-24T06:17:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.362+0000 7efcfbfff640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcfc0b3b10 con 0x7efcfc144770 2023-11-24T06:17:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.362+0000 7efcfbfff640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcfc0b43b0 con 0x7efcfc06d9e0 2023-11-24T06:17:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.362+0000 7efcfa7fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efcfc15f270 0x7efcfc0cd4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50650/0 (socket says 172.21.15.5:50650) 2023-11-24T06:17:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.362+0000 7efcfa7fc640 1 -- 172.21.15.5:0/858712315 learned_addr learned my addr 172.21.15.5:0/858712315 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:53.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.363+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/858712315 >> v2:172.21.15.6:3300/0 conn(0x7efcfc06d9e0 0x7efcfc165ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.363+0000 7efcf9ffb640 1 -- 172.21.15.5:0/858712315 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2459564951 0 0) 0x7efcfc15f930 con 0x7efcfc15f270 2023-11-24T06:17:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.363+0000 7efcf9ffb640 1 -- 172.21.15.5:0/858712315 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce4003520 con 0x7efcfc15f270 2023-11-24T06:17:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.363+0000 7efcf9ffb640 1 -- 172.21.15.5:0/858712315 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2627533561 0 0) 0x7efce4003520 con 0x7efcfc15f270 2023-11-24T06:17:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.363+0000 7efcf9ffb640 1 -- 172.21.15.5:0/858712315 >> v2:172.21.15.6:3300/0 conn(0x7efcfc06d9e0 msgr2=0x7efcfc165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.363+0000 7efcf9ffb640 1 --2- 172.21.15.5:0/858712315 >> v2:172.21.15.6:3300/0 conn(0x7efcfc06d9e0 0x7efcfc165ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.363+0000 7efcf9ffb640 1 -- 172.21.15.5:0/858712315 >> v1:172.21.15.6:6789/0 conn(0x7efcfc144770 legacy=0x7efcfc15eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.363+0000 7efcf9ffb640 1 -- 172.21.15.5:0/858712315 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc0b3df0 con 0x7efcfc15f270 2023-11-24T06:17:53.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcf9ffb640 1 -- 172.21.15.5:0/858712315 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7efcf0001690 con 0x7efcfc15f270 2023-11-24T06:17:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcf9ffb640 1 -- 172.21.15.5:0/858712315 >> v1:172.21.15.5:6789/0 conn(0x7efcfc15f270 legacy=0x7efcfc0cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcf9ffb640 1 --2- 172.21.15.5:0/858712315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce40039e0 0x7efce4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcf9ffb640 1 --2- 172.21.15.5:0/858712315 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efce40047c0 0x7efce4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcf9ffb640 1 --2- 172.21.15.5:0/858712315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efce4005180 0x7efce400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcf9ffb640 1 -- 172.21.15.5:0/858712315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0b43b0 con 0x7efce40039e0 2023-11-24T06:17:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcf9ffb640 1 -- 172.21.15.5:0/858712315 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce400c0b0 con 0x7efce4005180 2023-11-24T06:17:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcf9ffb640 1 -- 172.21.15.5:0/858712315 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce400c270 con 0x7efce40047c0 2023-11-24T06:17:53.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcfaffd640 1 --2- 172.21.15.5:0/858712315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce40039e0 0x7efce4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcfaffd640 1 -- 172.21.15.5:0/858712315 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efce40047c0 msgr2=0x7efce4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcfaffd640 1 --2- 172.21.15.5:0/858712315 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efce40047c0 0x7efce4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcfaffd640 1 -- 172.21.15.5:0/858712315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efce4005180 msgr2=0x7efce400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcfaffd640 1 --2- 172.21.15.5:0/858712315 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efce4005180 0x7efce400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcfaffd640 1 -- 172.21.15.5:0/858712315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce4003520 con 0x7efce40039e0 2023-11-24T06:17:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.364+0000 7efcfb7fe640 1 -- 172.21.15.5:0/858712315 reap_dead start 2023-11-24T06:17:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.365+0000 7efcfaffd640 1 --2- 172.21.15.5:0/858712315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce40039e0 0x7efce4003e00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efcec001750 tx=0x7efce40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be960b115f0bf83e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:53.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.366+0000 7efcf9ffb640 1 -- 172.21.15.5:0/858712315 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec0052d0 con 0x7efce40039e0 2023-11-24T06:17:53.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.366+0000 7efcf9ffb640 1 -- 172.21.15.5:0/858712315 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec02e050 con 0x7efce40039e0 2023-11-24T06:17:53.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.366+0000 7efcf9ffb640 1 -- 172.21.15.5:0/858712315 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec003830 con 0x7efce40039e0 2023-11-24T06:17:53.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.366+0000 7efcfbfff640 1 -- 172.21.15.5:0/858712315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce40039e0 msgr2=0x7efce4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:53.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.366+0000 7efcfbfff640 1 --2- 172.21.15.5:0/858712315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce40039e0 0x7efce4003e00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7efcec001750 tx=0x7efce40057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.366+0000 7efcfbfff640 1 -- 172.21.15.5:0/858712315 shutdown_connections 2023-11-24T06:17:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.366+0000 7efcfbfff640 1 --2- 172.21.15.5:0/858712315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce40039e0 0x7efce4003e00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.366+0000 7efcfbfff640 1 -- 172.21.15.5:0/858712315 >> 172.21.15.5:0/858712315 conn(0x7efcfc0cec10 msgr2=0x7efcfc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.367+0000 7efcfbfff640 1 -- 172.21.15.5:0/858712315 shutdown_connections 2023-11-24T06:17:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.367+0000 7efcfbfff640 1 -- 172.21.15.5:0/858712315 wait complete. 2023-11-24T06:17:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.368+0000 7efcfbfff640 1 Processor -- start 2023-11-24T06:17:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.368+0000 7efcfbfff640 1 -- start start 2023-11-24T06:17:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.369+0000 7efcfbfff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efce40039e0 0x7efcfc1432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.369+0000 7efcfbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc150780 0x7efcfc0aa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.369+0000 7efcfbfff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcfc0aad00 0x7efcfc1f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.369+0000 7efcfbfff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0ab180 con 0x7efcfc150780 2023-11-24T06:17:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.369+0000 7efcfbfff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc13bf40 con 0x7efcfc0aad00 2023-11-24T06:17:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.369+0000 7efcfbfff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc141b10 con 0x7efce40039e0 2023-11-24T06:17:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.370+0000 7efcfa7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc150780 0x7efcfc0aa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.370+0000 7efcfa7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc150780 0x7efcfc0aa7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58816/0 (socket says 172.21.15.5:58816) 2023-11-24T06:17:53.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.370+0000 7efcfa7fc640 1 -- 172.21.15.5:0/1391504912 learned_addr learned my addr 172.21.15.5:0/1391504912 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.370+0000 7efcfaffd640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efce40039e0 0x7efcfc1432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.370+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcfc0aad00 0x7efcfc1f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.370+0000 7efcfa7fc640 1 -- 172.21.15.5:0/1391504912 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efce40039e0 msgr2=0x7efcfc1432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.370+0000 7efcfa7fc640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efce40039e0 0x7efcfc1432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.370+0000 7efcfa7fc640 1 -- 172.21.15.5:0/1391504912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcfc0aad00 msgr2=0x7efcfc1f0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.370+0000 7efcfa7fc640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcfc0aad00 0x7efcfc1f0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.370+0000 7efcfa7fc640 1 -- 172.21.15.5:0/1391504912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc0a9be0 con 0x7efcfc150780 2023-11-24T06:17:53.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.370+0000 7efcfaffd640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7efce40039e0 0x7efcfc1432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.370+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7efcfc0aad00 0x7efcfc1f0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.370+0000 7efcfa7fc640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc150780 0x7efcfc0aa7c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efcf000c710 tx=0x7efcf000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.371+0000 7efce37fe640 1 -- 172.21.15.5:0/1391504912 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf00072d0 con 0x7efcfc150780 2023-11-24T06:17:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.371+0000 7efce37fe640 1 -- 172.21.15.5:0/1391504912 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf0007430 con 0x7efcfc150780 2023-11-24T06:17:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.371+0000 7efce37fe640 1 -- 172.21.15.5:0/1391504912 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf00047e0 con 0x7efcfc150780 2023-11-24T06:17:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.371+0000 7efcfbfff640 1 -- 172.21.15.5:0/1391504912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc141220 con 0x7efcfc150780 2023-11-24T06:17:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.371+0000 7efcfbfff640 1 -- 172.21.15.5:0/1391504912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc1417a0 con 0x7efcfc150780 2023-11-24T06:17:53.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.372+0000 7efcfbfff640 1 -- 172.21.15.5:0/1391504912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efcfc144b50 con 0x7efcfc150780 2023-11-24T06:17:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.373+0000 7efce37fe640 1 -- 172.21.15.5:0/1391504912 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7efcf0004940 con 0x7efcfc150780 2023-11-24T06:17:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.374+0000 7efce37fe640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efce806aed0 0x7efce806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.374+0000 7efce37fe640 1 -- 172.21.15.5:0/1391504912 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efcf0088850 con 0x7efcfc150780 2023-11-24T06:17:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.374+0000 7efcfaffd640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efce806aed0 0x7efce806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.374+0000 7efce37fe640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.5:6816/1571096651,v1:172.21.15.5:6817/1571096651] conn(0x7efce8070d30 0x7efce8073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.374+0000 7efce37fe640 1 -- 172.21.15.5:0/1391504912 --> [v2:172.21.15.5:6816/1571096651,v1:172.21.15.5:6817/1571096651] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efce8073800 con 0x7efce8070d30 2023-11-24T06:17:53.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.374+0000 7efce37fe640 1 -- 172.21.15.5:0/1391504912 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcf008ae30 con 0x7efcfc150780 2023-11-24T06:17:53.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.375+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.5:6816/1571096651,v1:172.21.15.5:6817/1571096651] conn(0x7efce8070d30 0x7efce8073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:53.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.375+0000 7efcfaffd640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efce806aed0 0x7efce806d390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efcec033000 tx=0x7efcec009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:53.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.375+0000 7efcfb7fe640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.5:6816/1571096651,v1:172.21.15.5:6817/1571096651] conn(0x7efce8070d30 0x7efce8073150 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:53.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.376+0000 7efce37fe640 1 -- 172.21.15.5:0/1391504912 <== osd.0 v2:172.21.15.5:6816/1571096651 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7efce8073800 con 0x7efce8070d30 2023-11-24T06:17:53.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.391+0000 7efcfbfff640 1 -- 172.21.15.5:0/1391504912 --> [v2:172.21.15.5:6816/1571096651,v1:172.21.15.5:6817/1571096651] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efcfc06c4b0 con 0x7efce8070d30 2023-11-24T06:17:53.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.391+0000 7efce37fe640 1 -- 172.21.15.5:0/1391504912 <== osd.0 v2:172.21.15.5:6816/1571096651 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7efcf4011040 con 0x7efce8070d30 2023-11-24T06:17:53.394 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-11-24T06:17:53.394 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:17:53.394 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-11-24T06:17:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.392+0000 7efcfbfff640 1 -- 172.21.15.5:0/1391504912 >> [v2:172.21.15.5:6816/1571096651,v1:172.21.15.5:6817/1571096651] conn(0x7efce8070d30 msgr2=0x7efce8073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.392+0000 7efcfbfff640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.5:6816/1571096651,v1:172.21.15.5:6817/1571096651] conn(0x7efce8070d30 0x7efce8073150 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.392+0000 7efcfbfff640 1 -- 172.21.15.5:0/1391504912 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efce806aed0 msgr2=0x7efce806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.392+0000 7efcfbfff640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7efce806aed0 0x7efce806d390 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efcec033000 tx=0x7efcec009030 comp rx=0 tx=0).stop 2023-11-24T06:17:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.392+0000 7efcfbfff640 1 -- 172.21.15.5:0/1391504912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc150780 msgr2=0x7efcfc0aa7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.392+0000 7efcfbfff640 1 --2- 172.21.15.5:0/1391504912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc150780 0x7efcfc0aa7c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7efcf000c710 tx=0x7efcf000cbd0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.392+0000 7efcfb7fe640 1 -- 172.21.15.5:0/1391504912 reap_dead start 2023-11-24T06:17:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.392+0000 7efcfbfff640 1 -- 172.21.15.5:0/1391504912 shutdown_connections 2023-11-24T06:17:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.392+0000 7efcfbfff640 1 -- 172.21.15.5:0/1391504912 >> 172.21.15.5:0/1391504912 conn(0x7efcfc0cec10 msgr2=0x7efcfc13edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.392+0000 7efcfbfff640 1 -- 172.21.15.5:0/1391504912 shutdown_connections 2023-11-24T06:17:53.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.392+0000 7efcfbfff640 1 -- 172.21.15.5:0/1391504912 wait complete. 2023-11-24T06:17:53.410 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-24T06:17:53.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.634+0000 7f2a64eff640 1 Processor -- start 2023-11-24T06:17:53.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.635+0000 7f2a64eff640 1 -- start start 2023-11-24T06:17:53.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.636+0000 7f2a64eff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a6015d430 0x7f2a600d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.636+0000 7f2a64eff640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f2a600d1690 0x7f2a600d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.636+0000 7f2a64eff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a600cff00 con 0x7f2a600cfaa0 2023-11-24T06:17:53.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.636+0000 7f2a64eff640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a600d09d0 con 0x7f2a600d1690 2023-11-24T06:17:53.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.636+0000 7f2a64eff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600d1030 con 0x7f2a6015d430 2023-11-24T06:17:53.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.637+0000 7f2a5effd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2a600cfaa0 0x7f2a601615e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50664/0 (socket says 172.21.15.5:50664) 2023-11-24T06:17:53.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.637+0000 7f2a5effd640 1 -- 172.21.15.5:0/1950738753 learned_addr learned my addr 172.21.15.5:0/1950738753 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:53.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.637+0000 7f2a5ffff640 1 --2- 172.21.15.5:0/1950738753 >> v2:172.21.15.6:3300/0 conn(0x7f2a600d1690 0x7f2a600d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:53.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.637+0000 7f2a5f7fe640 1 --2- 172.21.15.5:0/1950738753 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a6015d430 0x7f2a600d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:53.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.637+0000 7f2a5e7fc640 1 -- 172.21.15.5:0/1950738753 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052211876 0 0) 0x7f2a600cff00 con 0x7f2a600cfaa0 2023-11-24T06:17:53.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 -- 172.21.15.5:0/1950738753 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a4c003520 con 0x7f2a600cfaa0 2023-11-24T06:17:53.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 -- 172.21.15.5:0/1950738753 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 194757505 0 0) 0x7f2a4c003520 con 0x7f2a600cfaa0 2023-11-24T06:17:53.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 -- 172.21.15.5:0/1950738753 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a6015d430 msgr2=0x7f2a600d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:53.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 --2- 172.21.15.5:0/1950738753 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a6015d430 0x7f2a600d3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 -- 172.21.15.5:0/1950738753 >> v2:172.21.15.6:3300/0 conn(0x7f2a600d1690 msgr2=0x7f2a600d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:53.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 --2- 172.21.15.5:0/1950738753 >> v2:172.21.15.6:3300/0 conn(0x7f2a600d1690 0x7f2a600d1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 -- 172.21.15.5:0/1950738753 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a600d1fe0 con 0x7f2a600cfaa0 2023-11-24T06:17:53.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 -- 172.21.15.5:0/1950738753 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f2a48001670 con 0x7f2a600cfaa0 2023-11-24T06:17:53.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 -- 172.21.15.5:0/1950738753 >> v1:172.21.15.5:6789/0 conn(0x7f2a600cfaa0 legacy=0x7f2a601615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:53.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 --2- 172.21.15.5:0/1950738753 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a6015d430 0x7f2a4c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 --2- 172.21.15.5:0/1950738753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4c003fb0 0x7f2a4c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 --2- 172.21.15.5:0/1950738753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a4c004d00 0x7f2a4c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 -- 172.21.15.5:0/1950738753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600d09d0 con 0x7f2a4c003fb0 2023-11-24T06:17:53.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 -- 172.21.15.5:0/1950738753 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600d1030 con 0x7f2a4c004d00 2023-11-24T06:17:53.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.638+0000 7f2a5e7fc640 1 -- 172.21.15.5:0/1950738753 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c00ba80 con 0x7f2a6015d430 2023-11-24T06:17:53.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.639+0000 7f2a5effd640 1 --2- 172.21.15.5:0/1950738753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4c003fb0 0x7f2a4c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:53.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.639+0000 7f2a5f7fe640 1 --2- 172.21.15.5:0/1950738753 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a6015d430 0x7f2a4c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:53.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.639+0000 7f2a5ffff640 1 --2- 172.21.15.5:0/1950738753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a4c004d00 0x7f2a4c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:53.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.639+0000 7f2a5effd640 1 -- 172.21.15.5:0/1950738753 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a6015d430 msgr2=0x7f2a4c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:53.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.639+0000 7f2a5effd640 1 --2- 172.21.15.5:0/1950738753 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a6015d430 0x7f2a4c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.639+0000 7f2a5effd640 1 -- 172.21.15.5:0/1950738753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a4c004d00 msgr2=0x7f2a4c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:53.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.639+0000 7f2a5effd640 1 --2- 172.21.15.5:0/1950738753 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a4c004d00 0x7f2a4c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.639+0000 7f2a5effd640 1 -- 172.21.15.5:0/1950738753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c003520 con 0x7f2a4c003fb0 2023-11-24T06:17:53.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.640+0000 7f2a5effd640 1 --2- 172.21.15.5:0/1950738753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4c003fb0 0x7f2a4c0047c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c0e0 tx=0x7f2a4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fc9707944e2b6c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:53.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.641+0000 7f2a5e7fc640 1 -- 172.21.15.5:0/1950738753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48005ad0 con 0x7f2a4c003fb0 2023-11-24T06:17:53.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.641+0000 7f2a5e7fc640 1 -- 172.21.15.5:0/1950738753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a48005c30 con 0x7f2a4c003fb0 2023-11-24T06:17:53.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.641+0000 7f2a5e7fc640 1 -- 172.21.15.5:0/1950738753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4802c810 con 0x7f2a4c003fb0 2023-11-24T06:17:53.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.641+0000 7f2a64eff640 1 -- 172.21.15.5:0/1950738753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4c003fb0 msgr2=0x7f2a4c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:53.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.641+0000 7f2a64eff640 1 --2- 172.21.15.5:0/1950738753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4c003fb0 0x7f2a4c0047c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c0e0 tx=0x7f2a4c0057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.641+0000 7f2a5ffff640 1 -- 172.21.15.5:0/1950738753 reap_dead start 2023-11-24T06:17:53.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.641+0000 7f2a64eff640 1 -- 172.21.15.5:0/1950738753 shutdown_connections 2023-11-24T06:17:53.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.641+0000 7f2a64eff640 1 -- 172.21.15.5:0/1950738753 >> 172.21.15.5:0/1950738753 conn(0x7f2a6008cb00 msgr2=0x7f2a6009c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:53.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.641+0000 7f2a64eff640 1 -- 172.21.15.5:0/1950738753 shutdown_connections 2023-11-24T06:17:53.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.641+0000 7f2a64eff640 1 -- 172.21.15.5:0/1950738753 wait complete. 2023-11-24T06:17:53.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.643+0000 7f2a64eff640 1 Processor -- start 2023-11-24T06:17:53.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.643+0000 7f2a64eff640 1 -- start start 2023-11-24T06:17:53.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.644+0000 7f2a64eff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a4c003fb0 0x7f2a60152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.644+0000 7f2a64eff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a600c6be0 0x7f2a6015f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.644+0000 7f2a64eff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a600c7060 0x7f2a6015f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.644+0000 7f2a64eff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a60160830 con 0x7f2a600c7060 2023-11-24T06:17:53.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.644+0000 7f2a64eff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a60152750 con 0x7f2a4c003fb0 2023-11-24T06:17:53.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.644+0000 7f2a64eff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6015fd00 con 0x7f2a600c6be0 2023-11-24T06:17:53.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.645+0000 7f2a5ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a600c7060 0x7f2a6015f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.645+0000 7f2a5ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a600c7060 0x7f2a6015f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58830/0 (socket says 172.21.15.5:58830) 2023-11-24T06:17:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.645+0000 7f2a5ffff640 1 -- 172.21.15.5:0/3103694579 learned_addr learned my addr 172.21.15.5:0/3103694579 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.645+0000 7f2a5ffff640 1 -- 172.21.15.5:0/3103694579 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a600c6be0 msgr2=0x7f2a6015f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.645+0000 7f2a5ffff640 1 --2- 172.21.15.5:0/3103694579 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a600c6be0 0x7f2a6015f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.645+0000 7f2a5ffff640 1 -- 172.21.15.5:0/3103694579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a4c003fb0 msgr2=0x7f2a60152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.645+0000 7f2a5ffff640 1 --2- 172.21.15.5:0/3103694579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a4c003fb0 0x7f2a60152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.645+0000 7f2a5ffff640 1 -- 172.21.15.5:0/3103694579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a600b1730 con 0x7f2a600c7060 2023-11-24T06:17:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.645+0000 7f2a5ffff640 1 --2- 172.21.15.5:0/3103694579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a600c7060 0x7f2a6015f630 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2a6008d6f0 tx=0x7f2a58011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.645+0000 7f2a57fff640 1 -- 172.21.15.5:0/3103694579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a58004500 con 0x7f2a600c7060 2023-11-24T06:17:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.646+0000 7f2a57fff640 1 -- 172.21.15.5:0/3103694579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a58003610 con 0x7f2a600c7060 2023-11-24T06:17:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.646+0000 7f2a57fff640 1 -- 172.21.15.5:0/3103694579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a580039e0 con 0x7f2a600c7060 2023-11-24T06:17:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.646+0000 7f2a64eff640 1 -- 172.21.15.5:0/3103694579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a600b0e00 con 0x7f2a600c7060 2023-11-24T06:17:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.646+0000 7f2a64eff640 1 -- 172.21.15.5:0/3103694579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a600b13c0 con 0x7f2a600c7060 2023-11-24T06:17:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.647+0000 7f2a64eff640 1 -- 172.21.15.5:0/3103694579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a60067960 con 0x7f2a600c7060 2023-11-24T06:17:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.648+0000 7f2a57fff640 1 -- 172.21.15.5:0/3103694579 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f2a58003b40 con 0x7f2a600c7060 2023-11-24T06:17:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.649+0000 7f2a57fff640 1 --2- 172.21.15.5:0/3103694579 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2a4006aed0 0x7f2a4006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.649+0000 7f2a57fff640 1 -- 172.21.15.5:0/3103694579 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2a580898d0 con 0x7f2a600c7060 2023-11-24T06:17:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.649+0000 7f2a5f7fe640 1 --2- 172.21.15.5:0/3103694579 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2a4006aed0 0x7f2a4006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:53.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.650+0000 7f2a5f7fe640 1 --2- 172.21.15.5:0/3103694579 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2a4006aed0 0x7f2a4006d390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2a600cde40 tx=0x7f2a50009750 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:53.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.655+0000 7f2a57fff640 1 -- 172.21.15.5:0/3103694579 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a58054cd0 con 0x7f2a600c7060 2023-11-24T06:17:53.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.850+0000 7f2a64eff640 1 -- 172.21.15.5:0/3103694579 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a600d0410 con 0x7f2a4006aed0 2023-11-24T06:17:53.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.851+0000 7f2a57fff640 1 -- 172.21.15.5:0/3103694579 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2a5000f040 con 0x7f2a4006aed0 2023-11-24T06:17:53.853 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 0 to deep-scrub 2023-11-24T06:17:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.853+0000 7f2a64eff640 1 -- 172.21.15.5:0/3103694579 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2a4006aed0 msgr2=0x7f2a4006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.853+0000 7f2a64eff640 1 --2- 172.21.15.5:0/3103694579 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2a4006aed0 0x7f2a4006d390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2a600cde40 tx=0x7f2a50009750 comp rx=0 tx=0).stop 2023-11-24T06:17:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.853+0000 7f2a64eff640 1 -- 172.21.15.5:0/3103694579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a600c7060 msgr2=0x7f2a6015f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.853+0000 7f2a64eff640 1 --2- 172.21.15.5:0/3103694579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a600c7060 0x7f2a6015f630 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2a6008d6f0 tx=0x7f2a58011920 comp rx=0 tx=0).stop 2023-11-24T06:17:53.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.854+0000 7f2a64eff640 1 -- 172.21.15.5:0/3103694579 shutdown_connections 2023-11-24T06:17:53.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.854+0000 7f2a64eff640 1 --2- 172.21.15.5:0/3103694579 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2a4006aed0 0x7f2a4006d390 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.854+0000 7f2a64eff640 1 --2- 172.21.15.5:0/3103694579 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2a600c6be0 0x7f2a6015f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.854+0000 7f2a64eff640 1 --2- 172.21.15.5:0/3103694579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a600c7060 0x7f2a6015f630 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.854+0000 7f2a64eff640 1 --2- 172.21.15.5:0/3103694579 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2a4c003fb0 0x7f2a60152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.854+0000 7f2a64eff640 1 -- 172.21.15.5:0/3103694579 >> 172.21.15.5:0/3103694579 conn(0x7f2a6008cb00 msgr2=0x7f2a6006c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.854+0000 7f2a64eff640 1 -- 172.21.15.5:0/3103694579 shutdown_connections 2023-11-24T06:17:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:53.854+0000 7f2a64eff640 1 -- 172.21.15.5:0/3103694579 wait complete. 2023-11-24T06:17:53.875 INFO:tasks.ceph:Scrubbing osd.1 2023-11-24T06:17:53.875 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:17:54.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.101+0000 7f8804a09640 1 Processor -- start 2023-11-24T06:17:54.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.102+0000 7f8804a09640 1 -- start start 2023-11-24T06:17:54.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.103+0000 7f8804a09640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f88000b4140 0x7f8800071de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.103+0000 7f8804a09640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f88001439b0 0x7f8800070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.103+0000 7f8804a09640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8800070d40 con 0x7f88001435d0 2023-11-24T06:17:54.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.103+0000 7f8804a09640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8800071020 con 0x7f88001439b0 2023-11-24T06:17:54.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.103+0000 7f8804a09640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8800071300 con 0x7f88000b4140 2023-11-24T06:17:54.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.104+0000 7f87fffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f88001439b0 0x7f8800070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.104+0000 7f87fffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f88001439b0 0x7f8800070050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58846/0 (socket says 172.21.15.5:58846) 2023-11-24T06:17:54.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.104+0000 7f87fffff640 1 -- 172.21.15.5:0/802433548 learned_addr learned my addr 172.21.15.5:0/802433548 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:54.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.104+0000 7f87f7fff640 1 --2- 172.21.15.5:0/802433548 >> v2:172.21.15.6:3300/0 conn(0x7f88000b4140 0x7f8800071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.104+0000 7f87fffff640 1 -- 172.21.15.5:0/802433548 >> v2:172.21.15.6:3300/0 conn(0x7f88000b4140 msgr2=0x7f8800071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.104+0000 7f87fffff640 1 --2- 172.21.15.5:0/802433548 >> v2:172.21.15.6:3300/0 conn(0x7f88000b4140 0x7f8800071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.104+0000 7f87fffff640 1 -- 172.21.15.5:0/802433548 >> v1:172.21.15.6:6789/0 conn(0x7f88001435d0 legacy=0x7f88000b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.105+0000 7f87fffff640 1 -- 172.21.15.5:0/802433548 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88000715e0 con 0x7f88001439b0 2023-11-24T06:17:54.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.106+0000 7f87fffff640 1 --2- 172.21.15.5:0/802433548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88001439b0 0x7f8800070050 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f88000cbeb0 tx=0x7f87f802e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc25c08254a14d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.106+0000 7f87feffd640 1 -- 172.21.15.5:0/802433548 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f802ec00 con 0x7f88001439b0 2023-11-24T06:17:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.106+0000 7f87feffd640 1 -- 172.21.15.5:0/802433548 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f802ed60 con 0x7f88001439b0 2023-11-24T06:17:54.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.106+0000 7f87feffd640 1 -- 172.21.15.5:0/802433548 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8036610 con 0x7f88001439b0 2023-11-24T06:17:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.107+0000 7f8804a09640 1 -- 172.21.15.5:0/802433548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88001439b0 msgr2=0x7f8800070050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.107+0000 7f8804a09640 1 --2- 172.21.15.5:0/802433548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88001439b0 0x7f8800070050 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f88000cbeb0 tx=0x7f87f802e060 comp rx=0 tx=0).stop 2023-11-24T06:17:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.107+0000 7f8804a09640 1 -- 172.21.15.5:0/802433548 shutdown_connections 2023-11-24T06:17:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.107+0000 7f8804a09640 1 --2- 172.21.15.5:0/802433548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88001439b0 0x7f8800070050 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.107+0000 7f8804a09640 1 --2- 172.21.15.5:0/802433548 >> v2:172.21.15.6:3300/0 conn(0x7f88000b4140 0x7f8800071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.107+0000 7f8804a09640 1 -- 172.21.15.5:0/802433548 >> 172.21.15.5:0/802433548 conn(0x7f88000cec10 msgr2=0x7f88000cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:54.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.107+0000 7f8804a09640 1 -- 172.21.15.5:0/802433548 shutdown_connections 2023-11-24T06:17:54.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.107+0000 7f8804a09640 1 -- 172.21.15.5:0/802433548 wait complete. 2023-11-24T06:17:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.109+0000 7f8804a09640 1 Processor -- start 2023-11-24T06:17:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.109+0000 7f8804a09640 1 -- start start 2023-11-24T06:17:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.110+0000 7f8804a09640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88000b4140 0x7f880013feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.110+0000 7f8804a09640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88001435d0 0x7f880013ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.110+0000 7f8804a09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88001439b0 0x7f880015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.110+0000 7f8804a09640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88000cd2e0 con 0x7f88001439b0 2023-11-24T06:17:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.110+0000 7f8804a09640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88000cd530 con 0x7f88000b4140 2023-11-24T06:17:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.110+0000 7f8804a09640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88000cd810 con 0x7f88001435d0 2023-11-24T06:17:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.110+0000 7f87fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88001439b0 0x7f880015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.110+0000 7f87fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88001439b0 0x7f880015a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58860/0 (socket says 172.21.15.5:58860) 2023-11-24T06:17:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.110+0000 7f87fffff640 1 -- 172.21.15.5:0/2959665962 learned_addr learned my addr 172.21.15.5:0/2959665962 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:54.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.110+0000 7f87f7fff640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88001435d0 0x7f880013ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.110+0000 7f87ff7fe640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88000b4140 0x7f880013feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.110+0000 7f87fffff640 1 -- 172.21.15.5:0/2959665962 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88001435d0 msgr2=0x7f880013ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.110+0000 7f87fffff640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f88001435d0 0x7f880013ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.111+0000 7f87fffff640 1 -- 172.21.15.5:0/2959665962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88000b4140 msgr2=0x7f880013feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.111+0000 7f87fffff640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88000b4140 0x7f880013feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.111+0000 7f87fffff640 1 -- 172.21.15.5:0/2959665962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f8009b30 con 0x7f88001439b0 2023-11-24T06:17:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.111+0000 7f87ff7fe640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f88000b4140 0x7f880013feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.111+0000 7f87fffff640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88001439b0 0x7f880015a710 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f87f8038040 tx=0x7f87f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.111+0000 7f87fcff9640 1 -- 172.21.15.5:0/2959665962 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f802ec00 con 0x7f88001439b0 2023-11-24T06:17:54.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.111+0000 7f87fcff9640 1 -- 172.21.15.5:0/2959665962 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f8009510 con 0x7f88001439b0 2023-11-24T06:17:54.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.111+0000 7f87fcff9640 1 -- 172.21.15.5:0/2959665962 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f803f6b0 con 0x7f88001439b0 2023-11-24T06:17:54.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.111+0000 7f8804a09640 1 -- 172.21.15.5:0/2959665962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88000cdaf0 con 0x7f88001439b0 2023-11-24T06:17:54.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.111+0000 7f8804a09640 1 -- 172.21.15.5:0/2959665962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880013acf0 con 0x7f88001439b0 2023-11-24T06:17:54.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.112+0000 7f8804a09640 1 -- 172.21.15.5:0/2959665962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f880013aeb0 con 0x7f88001439b0 2023-11-24T06:17:54.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.114+0000 7f87fcff9640 1 -- 172.21.15.5:0/2959665962 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f87f803f810 con 0x7f88001439b0 2023-11-24T06:17:54.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.114+0000 7f87fcff9640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f87e006aed0 0x7f87e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.115+0000 7f87fcff9640 1 -- 172.21.15.5:0/2959665962 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f87f80abb80 con 0x7f88001439b0 2023-11-24T06:17:54.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.115+0000 7f87ff7fe640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f87e006aed0 0x7f87e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.115+0000 7f87fcff9640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.5:6800/1318127149,v1:172.21.15.5:6801/1318127149] conn(0x7f87e0070d30 0x7f87e0073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.115+0000 7f87fcff9640 1 -- 172.21.15.5:0/2959665962 --> [v2:172.21.15.5:6800/1318127149,v1:172.21.15.5:6801/1318127149] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f87e0073800 con 0x7f87e0070d30 2023-11-24T06:17:54.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.115+0000 7f87fcff9640 1 -- 172.21.15.5:0/2959665962 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87f80b02a0 con 0x7f88001439b0 2023-11-24T06:17:54.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.115+0000 7f87f7fff640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.5:6800/1318127149,v1:172.21.15.5:6801/1318127149] conn(0x7f87e0070d30 0x7f87e0073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.116+0000 7f87ff7fe640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f87e006aed0 0x7f87e006d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f87f0003eb0 tx=0x7f87f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:54.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.116+0000 7f87f7fff640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.5:6800/1318127149,v1:172.21.15.5:6801/1318127149] conn(0x7f87e0070d30 0x7f87e0073150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:54.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.117+0000 7f87fcff9640 1 -- 172.21.15.5:0/2959665962 <== osd.1 v2:172.21.15.5:6800/1318127149 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f87e0073800 con 0x7f87e0070d30 2023-11-24T06:17:54.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.131+0000 7f8804a09640 1 -- 172.21.15.5:0/2959665962 --> [v2:172.21.15.5:6800/1318127149,v1:172.21.15.5:6801/1318127149] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f880004e840 con 0x7f87e0070d30 2023-11-24T06:17:54.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.132+0000 7f87fcff9640 1 -- 172.21.15.5:0/2959665962 <== osd.1 v2:172.21.15.5:6800/1318127149 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f880004e840 con 0x7f87e0070d30 2023-11-24T06:17:54.134 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-11-24T06:17:54.134 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:17:54.134 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-11-24T06:17:54.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.132+0000 7f8804a09640 1 -- 172.21.15.5:0/2959665962 >> [v2:172.21.15.5:6800/1318127149,v1:172.21.15.5:6801/1318127149] conn(0x7f87e0070d30 msgr2=0x7f87e0073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.132+0000 7f8804a09640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.5:6800/1318127149,v1:172.21.15.5:6801/1318127149] conn(0x7f87e0070d30 0x7f87e0073150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.132+0000 7f8804a09640 1 -- 172.21.15.5:0/2959665962 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f87e006aed0 msgr2=0x7f87e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.132+0000 7f8804a09640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f87e006aed0 0x7f87e006d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f87f0003eb0 tx=0x7f87f0009040 comp rx=0 tx=0).stop 2023-11-24T06:17:54.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.132+0000 7f8804a09640 1 -- 172.21.15.5:0/2959665962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88001439b0 msgr2=0x7f880015a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.132+0000 7f8804a09640 1 --2- 172.21.15.5:0/2959665962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88001439b0 0x7f880015a710 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f87f8038040 tx=0x7f87f8004a40 comp rx=0 tx=0).stop 2023-11-24T06:17:54.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.132+0000 7f87fffff640 1 -- 172.21.15.5:0/2959665962 reap_dead start 2023-11-24T06:17:54.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.133+0000 7f8804a09640 1 -- 172.21.15.5:0/2959665962 shutdown_connections 2023-11-24T06:17:54.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.133+0000 7f8804a09640 1 -- 172.21.15.5:0/2959665962 >> 172.21.15.5:0/2959665962 conn(0x7f88000cec10 msgr2=0x7f88000ceff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:54.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.133+0000 7f8804a09640 1 -- 172.21.15.5:0/2959665962 shutdown_connections 2023-11-24T06:17:54.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.133+0000 7f8804a09640 1 -- 172.21.15.5:0/2959665962 wait complete. 2023-11-24T06:17:54.154 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-24T06:17:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.378+0000 7f2f5d29b640 1 Processor -- start 2023-11-24T06:17:54.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.378+0000 7f2f5d29b640 1 -- start start 2023-11-24T06:17:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.380+0000 7f2f5d29b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f580b39a0 0x7f2f58071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.380+0000 7f2f5d29b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f580abff0 0x7f2f58071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.380+0000 7f2f5d29b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f5813da30 con 0x7f2f5815a100 2023-11-24T06:17:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.380+0000 7f2f5d29b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f580ac840 con 0x7f2f580b39a0 2023-11-24T06:17:54.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.380+0000 7f2f5d29b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f580ac560 con 0x7f2f580abff0 2023-11-24T06:17:54.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.380+0000 7f2f4ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f580b39a0 0x7f2f58071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.380+0000 7f2f4ffff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2f580b39a0 0x7f2f58071130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58874/0 (socket says 172.21.15.5:58874) 2023-11-24T06:17:54.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.380+0000 7f2f4ffff640 1 -- 172.21.15.5:0/2532227428 learned_addr learned my addr 172.21.15.5:0/2532227428 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.381+0000 7f2f577fe640 1 --2- 172.21.15.5:0/2532227428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f580abff0 0x7f2f58071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.381+0000 7f2f4ffff640 1 -- 172.21.15.5:0/2532227428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f580abff0 msgr2=0x7f2f58071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.381+0000 7f2f4ffff640 1 --2- 172.21.15.5:0/2532227428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f580abff0 0x7f2f58071670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.381+0000 7f2f4ffff640 1 -- 172.21.15.5:0/2532227428 >> v1:172.21.15.6:6789/0 conn(0x7f2f5815a100 legacy=0x7f2f580b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.381+0000 7f2f4ffff640 1 -- 172.21.15.5:0/2532227428 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f58070650 con 0x7f2f580b39a0 2023-11-24T06:17:54.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.382+0000 7f2f4ffff640 1 --2- 172.21.15.5:0/2532227428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f580b39a0 0x7f2f58071130 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000bb10 tx=0x7f2f4002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbfc2958e01c60d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:54.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.382+0000 7f2f567fc640 1 -- 172.21.15.5:0/2532227428 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4002ee70 con 0x7f2f580b39a0 2023-11-24T06:17:54.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.382+0000 7f2f567fc640 1 -- 172.21.15.5:0/2532227428 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f40036440 con 0x7f2f580b39a0 2023-11-24T06:17:54.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.382+0000 7f2f567fc640 1 -- 172.21.15.5:0/2532227428 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f400367d0 con 0x7f2f580b39a0 2023-11-24T06:17:54.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.383+0000 7f2f5d29b640 1 -- 172.21.15.5:0/2532227428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f580b39a0 msgr2=0x7f2f58071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.383+0000 7f2f5d29b640 1 --2- 172.21.15.5:0/2532227428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f580b39a0 0x7f2f58071130 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000bb10 tx=0x7f2f4002e0c0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.383+0000 7f2f5d29b640 1 -- 172.21.15.5:0/2532227428 shutdown_connections 2023-11-24T06:17:54.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.383+0000 7f2f5d29b640 1 --2- 172.21.15.5:0/2532227428 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f580abff0 0x7f2f58071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.383+0000 7f2f5d29b640 1 --2- 172.21.15.5:0/2532227428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f580b39a0 0x7f2f58071130 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.383+0000 7f2f5d29b640 1 -- 172.21.15.5:0/2532227428 >> 172.21.15.5:0/2532227428 conn(0x7f2f580b0d80 msgr2=0x7f2f580b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:54.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.383+0000 7f2f5d29b640 1 -- 172.21.15.5:0/2532227428 shutdown_connections 2023-11-24T06:17:54.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.383+0000 7f2f5d29b640 1 -- 172.21.15.5:0/2532227428 wait complete. 2023-11-24T06:17:54.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.385+0000 7f2f5d29b640 1 Processor -- start 2023-11-24T06:17:54.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.385+0000 7f2f5d29b640 1 -- start start 2023-11-24T06:17:54.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f5d29b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f580abff0 0x7f2f5815ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f5d29b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f580b39a0 0x7f2f58154520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f5d29b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f5815a100 0x7f2f58154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f5d29b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5815d370 con 0x7f2f580b39a0 2023-11-24T06:17:54.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f5d29b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5815a700 con 0x7f2f5815a100 2023-11-24T06:17:54.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f5d29b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f580ce8f0 con 0x7f2f580abff0 2023-11-24T06:17:54.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f4ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f580b39a0 0x7f2f58154520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f4ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f580b39a0 0x7f2f58154520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58886/0 (socket says 172.21.15.5:58886) 2023-11-24T06:17:54.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f4ffff640 1 -- 172.21.15.5:0/105198969 learned_addr learned my addr 172.21.15.5:0/105198969 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:54.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f56ffd640 1 --2- 172.21.15.5:0/105198969 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f580abff0 0x7f2f5815ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f577fe640 1 --2- 172.21.15.5:0/105198969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f5815a100 0x7f2f58154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f4ffff640 1 -- 172.21.15.5:0/105198969 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f580abff0 msgr2=0x7f2f5815ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f4ffff640 1 --2- 172.21.15.5:0/105198969 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f580abff0 0x7f2f5815ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f4ffff640 1 -- 172.21.15.5:0/105198969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f5815a100 msgr2=0x7f2f58154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f4ffff640 1 --2- 172.21.15.5:0/105198969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f5815a100 0x7f2f58154a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.386+0000 7f2f4ffff640 1 -- 172.21.15.5:0/105198969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4000b770 con 0x7f2f580b39a0 2023-11-24T06:17:54.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.387+0000 7f2f4ffff640 1 --2- 172.21.15.5:0/105198969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f580b39a0 0x7f2f58154520 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000b2c0 tx=0x7f2f400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:54.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.387+0000 7f2f4f7fe640 1 -- 172.21.15.5:0/105198969 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40034b80 con 0x7f2f580b39a0 2023-11-24T06:17:54.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.387+0000 7f2f4f7fe640 1 -- 172.21.15.5:0/105198969 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f40034d00 con 0x7f2f580b39a0 2023-11-24T06:17:54.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.387+0000 7f2f4f7fe640 1 -- 172.21.15.5:0/105198969 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40041680 con 0x7f2f580b39a0 2023-11-24T06:17:54.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.387+0000 7f2f5d29b640 1 -- 172.21.15.5:0/105198969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5815a9a0 con 0x7f2f580b39a0 2023-11-24T06:17:54.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.387+0000 7f2f5d29b640 1 -- 172.21.15.5:0/105198969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f580cebd0 con 0x7f2f580b39a0 2023-11-24T06:17:54.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.388+0000 7f2f5d29b640 1 -- 172.21.15.5:0/105198969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f580cedc0 con 0x7f2f580b39a0 2023-11-24T06:17:54.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.389+0000 7f2f4f7fe640 1 -- 172.21.15.5:0/105198969 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f2f4003d040 con 0x7f2f580b39a0 2023-11-24T06:17:54.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.390+0000 7f2f4f7fe640 1 --2- 172.21.15.5:0/105198969 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f3806aed0 0x7f2f3806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.390+0000 7f2f4f7fe640 1 -- 172.21.15.5:0/105198969 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2f400ae450 con 0x7f2f580b39a0 2023-11-24T06:17:54.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.391+0000 7f2f56ffd640 1 --2- 172.21.15.5:0/105198969 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f3806aed0 0x7f2f3806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.391+0000 7f2f56ffd640 1 --2- 172.21.15.5:0/105198969 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f3806aed0 0x7f2f3806d390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2f48004ea0 tx=0x7f2f48009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:54.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.395+0000 7f2f4f7fe640 1 -- 172.21.15.5:0/105198969 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f580cedc0 con 0x7f2f580b39a0 2023-11-24T06:17:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.587+0000 7f2f5d29b640 1 -- 172.21.15.5:0/105198969 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f58070c10 con 0x7f2f3806aed0 2023-11-24T06:17:54.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.588+0000 7f2f4f7fe640 1 -- 172.21.15.5:0/105198969 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2f48005420 con 0x7f2f3806aed0 2023-11-24T06:17:54.590 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 1 to deep-scrub 2023-11-24T06:17:54.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.591+0000 7f2f5d29b640 1 -- 172.21.15.5:0/105198969 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f3806aed0 msgr2=0x7f2f3806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.591+0000 7f2f5d29b640 1 --2- 172.21.15.5:0/105198969 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f3806aed0 0x7f2f3806d390 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2f48004ea0 tx=0x7f2f48009040 comp rx=0 tx=0).stop 2023-11-24T06:17:54.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.591+0000 7f2f5d29b640 1 -- 172.21.15.5:0/105198969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f580b39a0 msgr2=0x7f2f58154520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.591+0000 7f2f5d29b640 1 --2- 172.21.15.5:0/105198969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f580b39a0 0x7f2f58154520 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000b2c0 tx=0x7f2f400046c0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.591+0000 7f2f5d29b640 1 -- 172.21.15.5:0/105198969 shutdown_connections 2023-11-24T06:17:54.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.591+0000 7f2f5d29b640 1 --2- 172.21.15.5:0/105198969 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f2f3806aed0 0x7f2f3806d390 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.591+0000 7f2f5d29b640 1 --2- 172.21.15.5:0/105198969 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f2f5815a100 0x7f2f58154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.591+0000 7f2f5d29b640 1 --2- 172.21.15.5:0/105198969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f580b39a0 0x7f2f58154520 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.591+0000 7f2f5d29b640 1 --2- 172.21.15.5:0/105198969 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f2f580abff0 0x7f2f5815ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.591+0000 7f2f5d29b640 1 -- 172.21.15.5:0/105198969 >> 172.21.15.5:0/105198969 conn(0x7f2f580b0d80 msgr2=0x7f2f580713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:54.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.591+0000 7f2f5d29b640 1 -- 172.21.15.5:0/105198969 shutdown_connections 2023-11-24T06:17:54.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.591+0000 7f2f5d29b640 1 -- 172.21.15.5:0/105198969 wait complete. 2023-11-24T06:17:54.610 INFO:tasks.ceph:Scrubbing osd.2 2023-11-24T06:17:54.611 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:17:54.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.830+0000 7fb81d29b640 1 Processor -- start 2023-11-24T06:17:54.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.831+0000 7fb81d29b640 1 -- start start 2023-11-24T06:17:54.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.833+0000 7fb81d29b640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fb8180ae270 0x7fb8180abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.833+0000 7fb81d29b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb81815b730 con 0x7fb81815b350 2023-11-24T06:17:54.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.833+0000 7fb81d29b640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8180ac740 con 0x7fb81809cb30 2023-11-24T06:17:54.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.833+0000 7fb81d29b640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8180ac460 con 0x7fb8180ae270 2023-11-24T06:17:54.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.833+0000 7fb816ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb81815b350 0x7fb8180b3530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50666/0 (socket says 172.21.15.5:50666) 2023-11-24T06:17:54.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.833+0000 7fb816ffd640 1 -- 172.21.15.5:0/1949435747 learned_addr learned my addr 172.21.15.5:0/1949435747 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:54.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.833+0000 7fb8177fe640 1 --2- 172.21.15.5:0/1949435747 >> v2:172.21.15.6:3300/0 conn(0x7fb8180ae270 0x7fb8180abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.833+0000 7fb815ffb640 1 -- 172.21.15.5:0/1949435747 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3258923833 0 0) 0x7fb808001110 con 0x7fb81815b350 2023-11-24T06:17:54.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.833+0000 7fb815ffb640 1 -- 172.21.15.5:0/1949435747 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb800003520 con 0x7fb81815b350 2023-11-24T06:17:54.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.834+0000 7fb815ffb640 1 -- 172.21.15.5:0/1949435747 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1658055802 0 0) 0x7fb800003520 con 0x7fb81815b350 2023-11-24T06:17:54.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.834+0000 7fb815ffb640 1 -- 172.21.15.5:0/1949435747 >> v2:172.21.15.6:3300/0 conn(0x7fb8180ae270 msgr2=0x7fb8180abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:54.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.834+0000 7fb815ffb640 1 --2- 172.21.15.5:0/1949435747 >> v2:172.21.15.6:3300/0 conn(0x7fb8180ae270 0x7fb8180abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.834+0000 7fb815ffb640 1 -- 172.21.15.5:0/1949435747 >> v1:172.21.15.6:6789/0 conn(0x7fb81809cb30 legacy=0x7fb8180adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.834+0000 7fb815ffb640 1 -- 172.21.15.5:0/1949435747 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8180b2a50 con 0x7fb81815b350 2023-11-24T06:17:54.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.834+0000 7fb815ffb640 1 -- 172.21.15.5:0/1949435747 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fb808001800 con 0x7fb81815b350 2023-11-24T06:17:54.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.834+0000 7fb815ffb640 1 -- 172.21.15.5:0/1949435747 >> v1:172.21.15.5:6789/0 conn(0x7fb81815b350 legacy=0x7fb8180b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.834+0000 7fb815ffb640 1 --2- 172.21.15.5:0/1949435747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8000039c0 0x7fb800003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.834+0000 7fb815ffb640 1 --2- 172.21.15.5:0/1949435747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8000047c0 0x7fb800004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.834+0000 7fb815ffb640 1 --2- 172.21.15.5:0/1949435747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb800005180 0x7fb80000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.834+0000 7fb815ffb640 1 -- 172.21.15.5:0/1949435747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180ac460 con 0x7fb800005180 2023-11-24T06:17:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.834+0000 7fb815ffb640 1 -- 172.21.15.5:0/1949435747 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80000c080 con 0x7fb8000047c0 2023-11-24T06:17:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.834+0000 7fb815ffb640 1 -- 172.21.15.5:0/1949435747 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80000c240 con 0x7fb8000039c0 2023-11-24T06:17:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.834+0000 7fb8177fe640 1 --2- 172.21.15.5:0/1949435747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb800005180 0x7fb80000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.835+0000 7fb8167fc640 1 --2- 172.21.15.5:0/1949435747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8000047c0 0x7fb800004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.835+0000 7fb8177fe640 1 -- 172.21.15.5:0/1949435747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8000039c0 msgr2=0x7fb800003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.835+0000 7fb8177fe640 1 --2- 172.21.15.5:0/1949435747 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8000039c0 0x7fb800003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.835+0000 7fb8177fe640 1 -- 172.21.15.5:0/1949435747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8000047c0 msgr2=0x7fb800004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:54.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.835+0000 7fb8177fe640 1 --2- 172.21.15.5:0/1949435747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8000047c0 0x7fb800004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.835+0000 7fb8177fe640 1 -- 172.21.15.5:0/1949435747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb800003520 con 0x7fb800005180 2023-11-24T06:17:54.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.835+0000 7fb8167fc640 1 --2- 172.21.15.5:0/1949435747 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb8000047c0 0x7fb800004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:54.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.836+0000 7fb8177fe640 1 -- 172.21.15.5:0/1949435747 reap_dead start 2023-11-24T06:17:54.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.836+0000 7fb8177fe640 1 --2- 172.21.15.5:0/1949435747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb800005180 0x7fb80000b980 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb810009fc0 tx=0x7fb8000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeae67c93d889b78 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:54.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.836+0000 7fb815ffb640 1 -- 172.21.15.5:0/1949435747 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81002cc60 con 0x7fb800005180 2023-11-24T06:17:54.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.836+0000 7fb815ffb640 1 -- 172.21.15.5:0/1949435747 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81000f040 con 0x7fb800005180 2023-11-24T06:17:54.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.836+0000 7fb815ffb640 1 -- 172.21.15.5:0/1949435747 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810003990 con 0x7fb800005180 2023-11-24T06:17:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.837+0000 7fb81d29b640 1 -- 172.21.15.5:0/1949435747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb800005180 msgr2=0x7fb80000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.837+0000 7fb81d29b640 1 --2- 172.21.15.5:0/1949435747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb800005180 0x7fb80000b980 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb810009fc0 tx=0x7fb8000057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.837+0000 7fb81d29b640 1 -- 172.21.15.5:0/1949435747 shutdown_connections 2023-11-24T06:17:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.837+0000 7fb81d29b640 1 --2- 172.21.15.5:0/1949435747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb800005180 0x7fb80000b980 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.837+0000 7fb81d29b640 1 -- 172.21.15.5:0/1949435747 >> 172.21.15.5:0/1949435747 conn(0x7fb8180b0ba0 msgr2=0x7fb8180b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.837+0000 7fb81d29b640 1 -- 172.21.15.5:0/1949435747 shutdown_connections 2023-11-24T06:17:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.837+0000 7fb81d29b640 1 -- 172.21.15.5:0/1949435747 wait complete. 2023-11-24T06:17:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.839+0000 7fb81d29b640 1 Processor -- start 2023-11-24T06:17:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.839+0000 7fb81d29b640 1 -- start start 2023-11-24T06:17:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb81d29b640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb800005180 0x7fb818149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb81d29b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb818151d30 0x7fb8181f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb81d29b640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8180c46d0 0x7fb8180714e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb81d29b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8181f1280 con 0x7fb818151d30 2023-11-24T06:17:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb81d29b640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb818149fa0 con 0x7fb800005180 2023-11-24T06:17:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb81d29b640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8181f0600 con 0x7fb8180c46d0 2023-11-24T06:17:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb8167fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb818151d30 0x7fb8181f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb8167fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb818151d30 0x7fb8181f16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58916/0 (socket says 172.21.15.5:58916) 2023-11-24T06:17:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb8167fc640 1 -- 172.21.15.5:0/808249387 learned_addr learned my addr 172.21.15.5:0/808249387 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb816ffd640 1 --2- 172.21.15.5:0/808249387 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb800005180 0x7fb818149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb8177fe640 1 --2- 172.21.15.5:0/808249387 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8180c46d0 0x7fb8180714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb8167fc640 1 -- 172.21.15.5:0/808249387 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8180c46d0 msgr2=0x7fb8180714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb8167fc640 1 --2- 172.21.15.5:0/808249387 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fb8180c46d0 0x7fb8180714e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb8167fc640 1 -- 172.21.15.5:0/808249387 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb800005180 msgr2=0x7fb818149a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb8167fc640 1 --2- 172.21.15.5:0/808249387 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fb800005180 0x7fb818149a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.840+0000 7fb8167fc640 1 -- 172.21.15.5:0/808249387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8181f08e0 con 0x7fb818151d30 2023-11-24T06:17:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.841+0000 7fb8167fc640 1 --2- 172.21.15.5:0/808249387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb818151d30 0x7fb8181f16a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00d560 tx=0x7fb80c00da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.841+0000 7fb7ff7fe640 1 -- 172.21.15.5:0/808249387 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c005480 con 0x7fb818151d30 2023-11-24T06:17:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.841+0000 7fb7ff7fe640 1 -- 172.21.15.5:0/808249387 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80c003610 con 0x7fb818151d30 2023-11-24T06:17:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.841+0000 7fb7ff7fe640 1 -- 172.21.15.5:0/808249387 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c003980 con 0x7fb818151d30 2023-11-24T06:17:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.841+0000 7fb81d29b640 1 -- 172.21.15.5:0/808249387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8181f0bc0 con 0x7fb818151d30 2023-11-24T06:17:54.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.841+0000 7fb81d29b640 1 -- 172.21.15.5:0/808249387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8181f1be0 con 0x7fb818151d30 2023-11-24T06:17:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.842+0000 7fb81d29b640 1 -- 172.21.15.5:0/808249387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb81806e720 con 0x7fb818151d30 2023-11-24T06:17:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.843+0000 7fb7ff7fe640 1 -- 172.21.15.5:0/808249387 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb80c003ae0 con 0x7fb818151d30 2023-11-24T06:17:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.844+0000 7fb7ff7fe640 1 --2- 172.21.15.5:0/808249387 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb80406aed0 0x7fb80406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.844+0000 7fb7ff7fe640 1 -- 172.21.15.5:0/808249387 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb80c089230 con 0x7fb818151d30 2023-11-24T06:17:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.844+0000 7fb7ff7fe640 1 --2- 172.21.15.5:0/808249387 >> [v2:172.21.15.5:6824/1782469371,v1:172.21.15.5:6825/1782469371] conn(0x7fb804070d30 0x7fb804073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.844+0000 7fb7ff7fe640 1 -- 172.21.15.5:0/808249387 --> [v2:172.21.15.5:6824/1782469371,v1:172.21.15.5:6825/1782469371] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb804073800 con 0x7fb804070d30 2023-11-24T06:17:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.844+0000 7fb7ff7fe640 1 -- 172.21.15.5:0/808249387 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb80c08b810 con 0x7fb818151d30 2023-11-24T06:17:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.845+0000 7fb816ffd640 1 --2- 172.21.15.5:0/808249387 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb80406aed0 0x7fb80406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.845+0000 7fb8177fe640 1 --2- 172.21.15.5:0/808249387 >> [v2:172.21.15.5:6824/1782469371,v1:172.21.15.5:6825/1782469371] conn(0x7fb804070d30 0x7fb804073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.845+0000 7fb816ffd640 1 --2- 172.21.15.5:0/808249387 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb80406aed0 0x7fb80406d390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb808001270 tx=0x7fb808001660 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.845+0000 7fb8177fe640 1 --2- 172.21.15.5:0/808249387 >> [v2:172.21.15.5:6824/1782469371,v1:172.21.15.5:6825/1782469371] conn(0x7fb804070d30 0x7fb804073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.846+0000 7fb7ff7fe640 1 -- 172.21.15.5:0/808249387 <== osd.2 v2:172.21.15.5:6824/1782469371 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb804073800 con 0x7fb804070d30 2023-11-24T06:17:54.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.861+0000 7fb81d29b640 1 -- 172.21.15.5:0/808249387 --> [v2:172.21.15.5:6824/1782469371,v1:172.21.15.5:6825/1782469371] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb8180b3010 con 0x7fb804070d30 2023-11-24T06:17:54.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.862+0000 7fb7ff7fe640 1 -- 172.21.15.5:0/808249387 <== osd.2 v2:172.21.15.5:6824/1782469371 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb81002cc00 con 0x7fb804070d30 2023-11-24T06:17:54.864 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-11-24T06:17:54.864 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:17:54.864 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-11-24T06:17:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.862+0000 7fb81d29b640 1 -- 172.21.15.5:0/808249387 >> [v2:172.21.15.5:6824/1782469371,v1:172.21.15.5:6825/1782469371] conn(0x7fb804070d30 msgr2=0x7fb804073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.862+0000 7fb81d29b640 1 --2- 172.21.15.5:0/808249387 >> [v2:172.21.15.5:6824/1782469371,v1:172.21.15.5:6825/1782469371] conn(0x7fb804070d30 0x7fb804073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:54.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.863+0000 7fb81d29b640 1 -- 172.21.15.5:0/808249387 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb80406aed0 msgr2=0x7fb80406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.863+0000 7fb81d29b640 1 --2- 172.21.15.5:0/808249387 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fb80406aed0 0x7fb80406d390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb808001270 tx=0x7fb808001660 comp rx=0 tx=0).stop 2023-11-24T06:17:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.863+0000 7fb81d29b640 1 -- 172.21.15.5:0/808249387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb818151d30 msgr2=0x7fb8181f16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.863+0000 7fb81d29b640 1 --2- 172.21.15.5:0/808249387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb818151d30 0x7fb8181f16a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00d560 tx=0x7fb80c00da20 comp rx=0 tx=0).stop 2023-11-24T06:17:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.863+0000 7fb8177fe640 1 -- 172.21.15.5:0/808249387 reap_dead start 2023-11-24T06:17:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.863+0000 7fb81d29b640 1 -- 172.21.15.5:0/808249387 shutdown_connections 2023-11-24T06:17:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.863+0000 7fb81d29b640 1 -- 172.21.15.5:0/808249387 >> 172.21.15.5:0/808249387 conn(0x7fb8180b0ba0 msgr2=0x7fb81809be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.863+0000 7fb81d29b640 1 -- 172.21.15.5:0/808249387 shutdown_connections 2023-11-24T06:17:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:54.863+0000 7fb81d29b640 1 -- 172.21.15.5:0/808249387 wait complete. 2023-11-24T06:17:54.882 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-24T06:17:55.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.110+0000 7f964b59e640 1 Processor -- start 2023-11-24T06:17:55.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.111+0000 7f964b59e640 1 -- start start 2023-11-24T06:17:55.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.112+0000 7f964b59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f964c15ed20 0x7f964c141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.112+0000 7f964b59e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f964c071820 0x7f964c06cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.112+0000 7f964b59e640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964c06cae0 con 0x7f964c15f100 2023-11-24T06:17:55.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.112+0000 7f964b59e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964c071e40 con 0x7f964c15ed20 2023-11-24T06:17:55.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.112+0000 7f964b59e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964c06c1b0 con 0x7f964c071820 2023-11-24T06:17:55.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.113+0000 7f964a59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f964c15ed20 0x7f964c141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.113+0000 7f964a59c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f964c15ed20 0x7f964c141e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58930/0 (socket says 172.21.15.5:58930) 2023-11-24T06:17:55.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.113+0000 7f964a59c640 1 -- 172.21.15.5:0/3716633795 learned_addr learned my addr 172.21.15.5:0/3716633795 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:55.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.113+0000 7f964a59c640 1 -- 172.21.15.5:0/3716633795 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f964c071820 msgr2=0x7f964c06cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:55.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.113+0000 7f964ad9d640 1 --2- 172.21.15.5:0/3716633795 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f964c071820 0x7f964c06cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.113+0000 7f964a59c640 1 --2- 172.21.15.5:0/3716633795 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f964c071820 0x7f964c06cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.113+0000 7f964a59c640 1 -- 172.21.15.5:0/3716633795 >> v1:172.21.15.6:6789/0 conn(0x7f964c15f100 legacy=0x7f964c071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:55.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.113+0000 7f964a59c640 1 -- 172.21.15.5:0/3716633795 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c06c490 con 0x7f964c15ed20 2023-11-24T06:17:55.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.113+0000 7f964ad9d640 1 --2- 172.21.15.5:0/3716633795 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f964c071820 0x7f964c06cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.114+0000 7f964a59c640 1 --2- 172.21.15.5:0/3716633795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964c15ed20 0x7f964c141e80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f963c009870 tx=0x7f963c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0fbde896bd4847 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.115+0000 7f9649d9b640 1 -- 172.21.15.5:0/3716633795 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c031640 con 0x7f964c15ed20 2023-11-24T06:17:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.115+0000 7f9649d9b640 1 -- 172.21.15.5:0/3716633795 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963c0317a0 con 0x7f964c15ed20 2023-11-24T06:17:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.115+0000 7f9649d9b640 1 -- 172.21.15.5:0/3716633795 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c031b30 con 0x7f964c15ed20 2023-11-24T06:17:55.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.115+0000 7f964b59e640 1 -- 172.21.15.5:0/3716633795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964c15ed20 msgr2=0x7f964c141e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.115+0000 7f964b59e640 1 --2- 172.21.15.5:0/3716633795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964c15ed20 0x7f964c141e80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f963c009870 tx=0x7f963c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.115+0000 7f964b59e640 1 -- 172.21.15.5:0/3716633795 shutdown_connections 2023-11-24T06:17:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.115+0000 7f964b59e640 1 --2- 172.21.15.5:0/3716633795 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f964c071820 0x7f964c06cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.115+0000 7f964b59e640 1 --2- 172.21.15.5:0/3716633795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964c15ed20 0x7f964c141e80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.115+0000 7f964b59e640 1 -- 172.21.15.5:0/3716633795 >> 172.21.15.5:0/3716633795 conn(0x7f964c0b0840 msgr2=0x7f964c0b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.115+0000 7f964b59e640 1 -- 172.21.15.5:0/3716633795 shutdown_connections 2023-11-24T06:17:55.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.115+0000 7f964b59e640 1 -- 172.21.15.5:0/3716633795 wait complete. 2023-11-24T06:17:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.117+0000 7f964b59e640 1 Processor -- start 2023-11-24T06:17:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.117+0000 7f964b59e640 1 -- start start 2023-11-24T06:17:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.118+0000 7f964b59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964c071820 0x7f964c1e8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.118+0000 7f964b59e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f964c15ed20 0x7f964c15e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.118+0000 7f964b59e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f964c15f100 0x7f964c15a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.118+0000 7f964b59e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c15e8a0 con 0x7f964c071820 2023-11-24T06:17:55.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.118+0000 7f964b59e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c1e9180 con 0x7f964c15f100 2023-11-24T06:17:55.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.118+0000 7f964b59e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964c1e9460 con 0x7f964c15ed20 2023-11-24T06:17:55.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.119+0000 7f964a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964c071820 0x7f964c1e8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.119+0000 7f964a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964c071820 0x7f964c1e8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58936/0 (socket says 172.21.15.5:58936) 2023-11-24T06:17:55.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.119+0000 7f964a59c640 1 -- 172.21.15.5:0/1756307143 learned_addr learned my addr 172.21.15.5:0/1756307143 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:55.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.119+0000 7f964a59c640 1 -- 172.21.15.5:0/1756307143 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f964c15ed20 msgr2=0x7f964c15e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:55.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.119+0000 7f964a59c640 1 --2- 172.21.15.5:0/1756307143 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f964c15ed20 0x7f964c15e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.119+0000 7f964a59c640 1 -- 172.21.15.5:0/1756307143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f964c15f100 msgr2=0x7f964c15a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:55.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.119+0000 7f964a59c640 1 --2- 172.21.15.5:0/1756307143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f964c15f100 0x7f964c15a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.119+0000 7f964a59c640 1 -- 172.21.15.5:0/1756307143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c0094d0 con 0x7f964c071820 2023-11-24T06:17:55.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.119+0000 7f964a59c640 1 --2- 172.21.15.5:0/1756307143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964c071820 0x7f964c1e8aa0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f963c009870 tx=0x7f963c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.120+0000 7f9642ffd640 1 -- 172.21.15.5:0/1756307143 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c03ac60 con 0x7f964c071820 2023-11-24T06:17:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.120+0000 7f9642ffd640 1 -- 172.21.15.5:0/1756307143 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963c03ade0 con 0x7f964c071820 2023-11-24T06:17:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.120+0000 7f9642ffd640 1 -- 172.21.15.5:0/1756307143 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963c044630 con 0x7f964c071820 2023-11-24T06:17:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.120+0000 7f964b59e640 1 -- 172.21.15.5:0/1756307143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c1e9740 con 0x7f964c071820 2023-11-24T06:17:55.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.120+0000 7f964b59e640 1 -- 172.21.15.5:0/1756307143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c1e9d00 con 0x7f964c071820 2023-11-24T06:17:55.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.121+0000 7f964b59e640 1 -- 172.21.15.5:0/1756307143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964c15f540 con 0x7f964c071820 2023-11-24T06:17:55.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.122+0000 7f9642ffd640 1 -- 172.21.15.5:0/1756307143 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f963c02f050 con 0x7f964c071820 2023-11-24T06:17:55.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.123+0000 7f9642ffd640 1 --2- 172.21.15.5:0/1756307143 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f962c06aed0 0x7f962c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.123+0000 7f9642ffd640 1 -- 172.21.15.5:0/1756307143 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f963c0ab640 con 0x7f964c071820 2023-11-24T06:17:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.124+0000 7f9643fff640 1 --2- 172.21.15.5:0/1756307143 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f962c06aed0 0x7f962c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.124+0000 7f9643fff640 1 --2- 172.21.15.5:0/1756307143 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f962c06aed0 0x7f962c06d390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9634006fd0 tx=0x7f9634006bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:55.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.127+0000 7f9642ffd640 1 -- 172.21.15.5:0/1756307143 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f964c15f540 con 0x7f964c071820 2023-11-24T06:17:55.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.320+0000 7f964b59e640 1 -- 172.21.15.5:0/1756307143 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f964c15f540 con 0x7f962c06aed0 2023-11-24T06:17:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.321+0000 7f9642ffd640 1 -- 172.21.15.5:0/1756307143 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f964c15f540 con 0x7f962c06aed0 2023-11-24T06:17:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 2 to deep-scrub 2023-11-24T06:17:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.324+0000 7f964b59e640 1 -- 172.21.15.5:0/1756307143 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f962c06aed0 msgr2=0x7f962c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.324+0000 7f964b59e640 1 --2- 172.21.15.5:0/1756307143 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f962c06aed0 0x7f962c06d390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9634006fd0 tx=0x7f9634006bc0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.324+0000 7f964b59e640 1 -- 172.21.15.5:0/1756307143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964c071820 msgr2=0x7f964c1e8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:55.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.324+0000 7f964b59e640 1 --2- 172.21.15.5:0/1756307143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964c071820 0x7f964c1e8aa0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f963c009870 tx=0x7f963c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.324+0000 7f964b59e640 1 -- 172.21.15.5:0/1756307143 shutdown_connections 2023-11-24T06:17:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.324+0000 7f964b59e640 1 --2- 172.21.15.5:0/1756307143 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f962c06aed0 0x7f962c06d390 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.324+0000 7f964b59e640 1 --2- 172.21.15.5:0/1756307143 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f964c15ed20 0x7f964c15e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.324+0000 7f964b59e640 1 --2- 172.21.15.5:0/1756307143 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f964c15f100 0x7f964c15a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.324+0000 7f964b59e640 1 --2- 172.21.15.5:0/1756307143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f964c071820 0x7f964c1e8aa0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.324+0000 7f964b59e640 1 -- 172.21.15.5:0/1756307143 >> 172.21.15.5:0/1756307143 conn(0x7f964c0b0840 msgr2=0x7f964c0aec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.324+0000 7f964b59e640 1 -- 172.21.15.5:0/1756307143 shutdown_connections 2023-11-24T06:17:55.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.325+0000 7f964b59e640 1 -- 172.21.15.5:0/1756307143 wait complete. 2023-11-24T06:17:55.344 INFO:tasks.ceph:Scrubbing osd.3 2023-11-24T06:17:55.345 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:17:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.574+0000 7f360181a640 1 Processor -- start 2023-11-24T06:17:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.574+0000 7f360181a640 1 -- start start 2023-11-24T06:17:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.576+0000 7f360181a640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f35fc153ea0 0x7f35fc0ad450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.576+0000 7f360181a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f35fc070850 0x7f35fc071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.576+0000 7f360181a640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35fc0ce650 con 0x7f35fc154280 2023-11-24T06:17:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.576+0000 7f360181a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc154710 con 0x7f35fc070850 2023-11-24T06:17:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.576+0000 7f360181a640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35fc15e0a0 con 0x7f35fc153ea0 2023-11-24T06:17:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.577+0000 7f3601019640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f35fc070850 0x7f35fc071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.577+0000 7f3601019640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f35fc070850 0x7f35fc071350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58938/0 (socket says 172.21.15.5:58938) 2023-11-24T06:17:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.577+0000 7f3601019640 1 -- 172.21.15.5:0/1808027785 learned_addr learned my addr 172.21.15.5:0/1808027785 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.577+0000 7f3600818640 1 --2- 172.21.15.5:0/1808027785 >> v2:172.21.15.6:3300/0 conn(0x7f35fc153ea0 0x7f35fc0ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.577+0000 7f3601019640 1 -- 172.21.15.5:0/1808027785 >> v2:172.21.15.6:3300/0 conn(0x7f35fc153ea0 msgr2=0x7f35fc0ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.577+0000 7f3601019640 1 --2- 172.21.15.5:0/1808027785 >> v2:172.21.15.6:3300/0 conn(0x7f35fc153ea0 0x7f35fc0ad450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.577+0000 7f3601019640 1 -- 172.21.15.5:0/1808027785 >> v1:172.21.15.6:6789/0 conn(0x7f35fc154280 legacy=0x7f35fc15d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.577+0000 7f3601019640 1 -- 172.21.15.5:0/1808027785 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35fc15e380 con 0x7f35fc070850 2023-11-24T06:17:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.579+0000 7f3601019640 1 --2- 172.21.15.5:0/1808027785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35fc070850 0x7f35fc071350 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0cfc10 tx=0x7f35f402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c2274ff83e099e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.579+0000 7f35fb7fe640 1 -- 172.21.15.5:0/1808027785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f402ca80 con 0x7f35fc070850 2023-11-24T06:17:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.579+0000 7f35fb7fe640 1 -- 172.21.15.5:0/1808027785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f402cbe0 con 0x7f35fc070850 2023-11-24T06:17:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.579+0000 7f35fb7fe640 1 -- 172.21.15.5:0/1808027785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40343f0 con 0x7f35fc070850 2023-11-24T06:17:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.580+0000 7f360181a640 1 -- 172.21.15.5:0/1808027785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35fc070850 msgr2=0x7f35fc071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.580+0000 7f360181a640 1 --2- 172.21.15.5:0/1808027785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35fc070850 0x7f35fc071350 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0cfc10 tx=0x7f35f402bee0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.580+0000 7f360181a640 1 -- 172.21.15.5:0/1808027785 shutdown_connections 2023-11-24T06:17:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.580+0000 7f360181a640 1 --2- 172.21.15.5:0/1808027785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35fc070850 0x7f35fc071350 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.580+0000 7f360181a640 1 --2- 172.21.15.5:0/1808027785 >> v2:172.21.15.6:3300/0 conn(0x7f35fc153ea0 0x7f35fc0ad450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.580+0000 7f360181a640 1 -- 172.21.15.5:0/1808027785 >> 172.21.15.5:0/1808027785 conn(0x7f35fc0cec10 msgr2=0x7f35fc0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.580+0000 7f360181a640 1 -- 172.21.15.5:0/1808027785 shutdown_connections 2023-11-24T06:17:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.580+0000 7f360181a640 1 -- 172.21.15.5:0/1808027785 wait complete. 2023-11-24T06:17:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.582+0000 7f360181a640 1 Processor -- start 2023-11-24T06:17:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.582+0000 7f360181a640 1 -- start start 2023-11-24T06:17:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.583+0000 7f360181a640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f35fc070850 0x7f35fc144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.583+0000 7f360181a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35fc153ea0 0x7f35fc14ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.583+0000 7f360181a640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35fc154280 0x7f35fc14fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.583+0000 7f360181a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc154fb0 con 0x7f35fc153ea0 2023-11-24T06:17:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.583+0000 7f360181a640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc144fe0 con 0x7f35fc154280 2023-11-24T06:17:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.583+0000 7f360181a640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35fc14fa60 con 0x7f35fc070850 2023-11-24T06:17:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.583+0000 7f35fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35fc153ea0 0x7f35fc14ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.584+0000 7f35fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35fc153ea0 0x7f35fc14ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58942/0 (socket says 172.21.15.5:58942) 2023-11-24T06:17:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.584+0000 7f35fbfff640 1 -- 172.21.15.5:0/1478365181 learned_addr learned my addr 172.21.15.5:0/1478365181 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.584+0000 7f3600818640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f35fc070850 0x7f35fc144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.584+0000 7f3601019640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35fc154280 0x7f35fc14fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.584+0000 7f35fbfff640 1 -- 172.21.15.5:0/1478365181 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f35fc070850 msgr2=0x7f35fc144aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.584+0000 7f35fbfff640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f35fc070850 0x7f35fc144aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.584+0000 7f35fbfff640 1 -- 172.21.15.5:0/1478365181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35fc154280 msgr2=0x7f35fc14fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:55.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.584+0000 7f35fbfff640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35fc154280 0x7f35fc14fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.584+0000 7f35fbfff640 1 -- 172.21.15.5:0/1478365181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f4009b30 con 0x7f35fc153ea0 2023-11-24T06:17:55.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.584+0000 7f3600818640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f35fc070850 0x7f35fc144aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:55.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.584+0000 7f3601019640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f35fc154280 0x7f35fc14fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:55.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.584+0000 7f35fbfff640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35fc153ea0 0x7f35fc14ece0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f35e800b420 tx=0x7f35e800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:55.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.585+0000 7f35f97fa640 1 -- 172.21.15.5:0/1478365181 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8011830 con 0x7f35fc153ea0 2023-11-24T06:17:55.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.585+0000 7f35f97fa640 1 -- 172.21.15.5:0/1478365181 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e8011990 con 0x7f35fc153ea0 2023-11-24T06:17:55.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.585+0000 7f35f97fa640 1 -- 172.21.15.5:0/1478365181 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e8011d40 con 0x7f35fc153ea0 2023-11-24T06:17:55.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.585+0000 7f360181a640 1 -- 172.21.15.5:0/1478365181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35fc14f780 con 0x7f35fc153ea0 2023-11-24T06:17:55.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.585+0000 7f360181a640 1 -- 172.21.15.5:0/1478365181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35fc154bd0 con 0x7f35fc153ea0 2023-11-24T06:17:55.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.586+0000 7f360181a640 1 -- 172.21.15.5:0/1478365181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35fc070330 con 0x7f35fc153ea0 2023-11-24T06:17:55.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.587+0000 7f35f97fa640 1 -- 172.21.15.5:0/1478365181 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f35e801b460 con 0x7f35fc153ea0 2023-11-24T06:17:55.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.588+0000 7f35f97fa640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f35e006aed0 0x7f35e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.588+0000 7f35f97fa640 1 -- 172.21.15.5:0/1478365181 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f35e8088fe0 con 0x7f35fc153ea0 2023-11-24T06:17:55.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.588+0000 7f35f97fa640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.5:6808/337602632,v1:172.21.15.5:6809/337602632] conn(0x7f35e0070d30 0x7f35e0073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.588+0000 7f3600818640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f35e006aed0 0x7f35e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.588+0000 7f35f97fa640 1 -- 172.21.15.5:0/1478365181 --> [v2:172.21.15.5:6808/337602632,v1:172.21.15.5:6809/337602632] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35e0073800 con 0x7f35e0070d30 2023-11-24T06:17:55.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.588+0000 7f35f97fa640 1 -- 172.21.15.5:0/1478365181 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35e808b600 con 0x7f35fc153ea0 2023-11-24T06:17:55.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.588+0000 7f3601019640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.5:6808/337602632,v1:172.21.15.5:6809/337602632] conn(0x7f35e0070d30 0x7f35e0073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.589+0000 7f3600818640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f35e006aed0 0x7f35e006d390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009ac0 tx=0x7f35f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:55.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.589+0000 7f3601019640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.5:6808/337602632,v1:172.21.15.5:6809/337602632] conn(0x7f35e0070d30 0x7f35e0073150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:55.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.590+0000 7f35f97fa640 1 -- 172.21.15.5:0/1478365181 <== osd.3 v2:172.21.15.5:6808/337602632 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f35e0073800 con 0x7f35e0070d30 2023-11-24T06:17:55.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.605+0000 7f360181a640 1 -- 172.21.15.5:0/1478365181 --> [v2:172.21.15.5:6808/337602632,v1:172.21.15.5:6809/337602632] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f35fc070050 con 0x7f35e0070d30 2023-11-24T06:17:55.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.605+0000 7f35f97fa640 1 -- 172.21.15.5:0/1478365181 <== osd.3 v2:172.21.15.5:6808/337602632 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f35f402ca50 con 0x7f35e0070d30 2023-11-24T06:17:55.610 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-11-24T06:17:55.610 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:17:55.610 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-11-24T06:17:55.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.606+0000 7f360181a640 1 -- 172.21.15.5:0/1478365181 >> [v2:172.21.15.5:6808/337602632,v1:172.21.15.5:6809/337602632] conn(0x7f35e0070d30 msgr2=0x7f35e0073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:55.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.606+0000 7f360181a640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.5:6808/337602632,v1:172.21.15.5:6809/337602632] conn(0x7f35e0070d30 0x7f35e0073150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.606+0000 7f360181a640 1 -- 172.21.15.5:0/1478365181 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f35e006aed0 msgr2=0x7f35e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:55.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.606+0000 7f360181a640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f35e006aed0 0x7f35e006d390 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009ac0 tx=0x7f35f0008040 comp rx=0 tx=0).stop 2023-11-24T06:17:55.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.606+0000 7f360181a640 1 -- 172.21.15.5:0/1478365181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35fc153ea0 msgr2=0x7f35fc14ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:55.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.606+0000 7f360181a640 1 --2- 172.21.15.5:0/1478365181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35fc153ea0 0x7f35fc14ece0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f35e800b420 tx=0x7f35e800b8e0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.606+0000 7f3601019640 1 -- 172.21.15.5:0/1478365181 reap_dead start 2023-11-24T06:17:55.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.606+0000 7f360181a640 1 -- 172.21.15.5:0/1478365181 shutdown_connections 2023-11-24T06:17:55.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.606+0000 7f360181a640 1 -- 172.21.15.5:0/1478365181 >> 172.21.15.5:0/1478365181 conn(0x7f35fc0cec10 msgr2=0x7f35fc0ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:55.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.606+0000 7f360181a640 1 -- 172.21.15.5:0/1478365181 shutdown_connections 2023-11-24T06:17:55.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.606+0000 7f360181a640 1 -- 172.21.15.5:0/1478365181 wait complete. 2023-11-24T06:17:55.624 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-24T06:17:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.836+0000 7f31beab4640 1 Processor -- start 2023-11-24T06:17:55.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.836+0000 7f31beab4640 1 -- start start 2023-11-24T06:17:55.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.838+0000 7f31beab4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f31b815d540 0x7f31b8070fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.838+0000 7f31beab4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f31b80cad30 0x7f31b8160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.838+0000 7f31beab4640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f31b8161460 0x7f31b80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.838+0000 7f31beab4640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b8161d80 con 0x7f31b815d540 2023-11-24T06:17:55.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.838+0000 7f31beab4640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b80cb1c0 con 0x7f31b8161460 2023-11-24T06:17:55.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.838+0000 7f31beab4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b81618b0 con 0x7f31b80cad30 2023-11-24T06:17:55.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.838+0000 7f31bdab2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f31b815d540 0x7f31b8070fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.838+0000 7f31bdab2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f31b815d540 0x7f31b8070fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58944/0 (socket says 172.21.15.5:58944) 2023-11-24T06:17:55.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.838+0000 7f31bdab2640 1 -- 172.21.15.5:0/2865722897 learned_addr learned my addr 172.21.15.5:0/2865722897 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.839+0000 7f31bd2b1640 1 --2- 172.21.15.5:0/2865722897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f31b80cad30 0x7f31b8160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.839+0000 7f31be2b3640 1 --2- 172.21.15.5:0/2865722897 >> v2:172.21.15.6:3300/0 conn(0x7f31b8161460 0x7f31b80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.839+0000 7f31bdab2640 1 -- 172.21.15.5:0/2865722897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f31b80cad30 msgr2=0x7f31b8160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.839+0000 7f31bdab2640 1 --2- 172.21.15.5:0/2865722897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f31b80cad30 0x7f31b8160f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.839+0000 7f31bdab2640 1 -- 172.21.15.5:0/2865722897 >> v2:172.21.15.6:3300/0 conn(0x7f31b8161460 msgr2=0x7f31b80b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:55.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.839+0000 7f31bdab2640 1 --2- 172.21.15.5:0/2865722897 >> v2:172.21.15.6:3300/0 conn(0x7f31b8161460 0x7f31b80b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.839+0000 7f31bdab2640 1 -- 172.21.15.5:0/2865722897 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b8071bc0 con 0x7f31b815d540 2023-11-24T06:17:55.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.840+0000 7f31bdab2640 1 --2- 172.21.15.5:0/2865722897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31b815d540 0x7f31b8070fd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f31b80cfc70 tx=0x7f31b402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1b13aeb8ebafc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.841+0000 7f31bcab0640 1 -- 172.21.15.5:0/2865722897 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4031750 con 0x7f31b815d540 2023-11-24T06:17:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.841+0000 7f31bcab0640 1 -- 172.21.15.5:0/2865722897 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b40318b0 con 0x7f31b815d540 2023-11-24T06:17:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.841+0000 7f31bcab0640 1 -- 172.21.15.5:0/2865722897 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b4031c40 con 0x7f31b815d540 2023-11-24T06:17:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.841+0000 7f31beab4640 1 -- 172.21.15.5:0/2865722897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31b815d540 msgr2=0x7f31b8070fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.841+0000 7f31beab4640 1 --2- 172.21.15.5:0/2865722897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31b815d540 0x7f31b8070fd0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f31b80cfc70 tx=0x7f31b402b7f0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.841+0000 7f31beab4640 1 -- 172.21.15.5:0/2865722897 shutdown_connections 2023-11-24T06:17:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.841+0000 7f31beab4640 1 --2- 172.21.15.5:0/2865722897 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f31b80cad30 0x7f31b8160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.842+0000 7f31beab4640 1 --2- 172.21.15.5:0/2865722897 >> v2:172.21.15.6:3300/0 conn(0x7f31b8161460 0x7f31b80b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.842+0000 7f31beab4640 1 --2- 172.21.15.5:0/2865722897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31b815d540 0x7f31b8070fd0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.842+0000 7f31beab4640 1 -- 172.21.15.5:0/2865722897 >> 172.21.15.5:0/2865722897 conn(0x7f31b80b0840 msgr2=0x7f31b80b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.842+0000 7f31beab4640 1 -- 172.21.15.5:0/2865722897 shutdown_connections 2023-11-24T06:17:55.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.842+0000 7f31beab4640 1 -- 172.21.15.5:0/2865722897 wait complete. 2023-11-24T06:17:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.844+0000 7f31beab4640 1 Processor -- start 2023-11-24T06:17:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.844+0000 7f31beab4640 1 -- start start 2023-11-24T06:17:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.845+0000 7f31beab4640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b80cad30 0x7f31b814f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.845+0000 7f31beab4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31b8161460 0x7f31b80cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.845+0000 7f31beab4640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f31b80cb840 0x7f31b81e8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.845+0000 7f31beab4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b814fbf0 con 0x7f31b8161460 2023-11-24T06:17:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.845+0000 7f31beab4640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b815e260 con 0x7f31b80cad30 2023-11-24T06:17:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.845+0000 7f31beab4640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b80cbd50 con 0x7f31b80cb840 2023-11-24T06:17:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.845+0000 7f31bd2b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31b8161460 0x7f31b80cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.845+0000 7f31bd2b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31b8161460 0x7f31b80cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58952/0 (socket says 172.21.15.5:58952) 2023-11-24T06:17:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.845+0000 7f31bd2b1640 1 -- 172.21.15.5:0/3382907772 learned_addr learned my addr 172.21.15.5:0/3382907772 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.846+0000 7f31be2b3640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f31b80cb840 0x7f31b81e8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.846+0000 7f31bdab2640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b80cad30 0x7f31b814f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.846+0000 7f31bd2b1640 1 -- 172.21.15.5:0/3382907772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f31b80cb840 msgr2=0x7f31b81e8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.846+0000 7f31bd2b1640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f31b80cb840 0x7f31b81e8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.846+0000 7f31bd2b1640 1 -- 172.21.15.5:0/3382907772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b80cad30 msgr2=0x7f31b814f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.846+0000 7f31bd2b1640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b80cad30 0x7f31b814f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.846+0000 7f31bd2b1640 1 -- 172.21.15.5:0/3382907772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b40094d0 con 0x7f31b8161460 2023-11-24T06:17:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.846+0000 7f31be2b3640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f31b80cb840 0x7f31b81e8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.846+0000 7f31bd2b1640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31b8161460 0x7f31b80cb300 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f31a800d800 tx=0x7f31a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.847+0000 7f31ae7fc640 1 -- 172.21.15.5:0/3382907772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8009640 con 0x7f31b8161460 2023-11-24T06:17:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.847+0000 7f31ae7fc640 1 -- 172.21.15.5:0/3382907772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a80097a0 con 0x7f31b8161460 2023-11-24T06:17:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.847+0000 7f31ae7fc640 1 -- 172.21.15.5:0/3382907772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8009930 con 0x7f31b8161460 2023-11-24T06:17:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.847+0000 7f31beab4640 1 -- 172.21.15.5:0/3382907772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b815d930 con 0x7f31b8161460 2023-11-24T06:17:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.847+0000 7f31beab4640 1 -- 172.21.15.5:0/3382907772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b815def0 con 0x7f31b8161460 2023-11-24T06:17:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.848+0000 7f31beab4640 1 -- 172.21.15.5:0/3382907772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b804e840 con 0x7f31b8161460 2023-11-24T06:17:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.849+0000 7f31ae7fc640 1 -- 172.21.15.5:0/3382907772 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f31a8009a90 con 0x7f31b8161460 2023-11-24T06:17:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.850+0000 7f31ae7fc640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f319c06aed0 0x7f319c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:55.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.850+0000 7f31ae7fc640 1 -- 172.21.15.5:0/3382907772 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f31a80897b0 con 0x7f31b8161460 2023-11-24T06:17:55.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.850+0000 7f31bdab2640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f319c06aed0 0x7f319c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:55.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.851+0000 7f31bdab2640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f319c06aed0 0x7f319c06d390 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f31b4035000 tx=0x7f31b4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:55.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:55.855+0000 7f31ae7fc640 1 -- 172.21.15.5:0/3382907772 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f31a8054b70 con 0x7f31b8161460 2023-11-24T06:17:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.046+0000 7f31beab4640 1 -- 172.21.15.5:0/3382907772 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f31b806c790 con 0x7f319c06aed0 2023-11-24T06:17:56.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.047+0000 7f31ae7fc640 1 -- 172.21.15.5:0/3382907772 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f31b4031af0 con 0x7f319c06aed0 2023-11-24T06:17:56.049 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 3 to deep-scrub 2023-11-24T06:17:56.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.050+0000 7f31beab4640 1 -- 172.21.15.5:0/3382907772 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f319c06aed0 msgr2=0x7f319c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:56.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.050+0000 7f31beab4640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f319c06aed0 0x7f319c06d390 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f31b4035000 tx=0x7f31b4008030 comp rx=0 tx=0).stop 2023-11-24T06:17:56.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.050+0000 7f31beab4640 1 -- 172.21.15.5:0/3382907772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31b8161460 msgr2=0x7f31b80cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:56.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.050+0000 7f31beab4640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31b8161460 0x7f31b80cb300 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f31a800d800 tx=0x7f31a800dcc0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.050+0000 7f31beab4640 1 -- 172.21.15.5:0/3382907772 shutdown_connections 2023-11-24T06:17:56.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.050+0000 7f31beab4640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f319c06aed0 0x7f319c06d390 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.050+0000 7f31beab4640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f31b80cb840 0x7f31b81e8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.050+0000 7f31beab4640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31b8161460 0x7f31b80cb300 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.050+0000 7f31beab4640 1 --2- 172.21.15.5:0/3382907772 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f31b80cad30 0x7f31b814f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.050+0000 7f31beab4640 1 -- 172.21.15.5:0/3382907772 >> 172.21.15.5:0/3382907772 conn(0x7f31b80b0840 msgr2=0x7f31b80af300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:56.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.050+0000 7f31beab4640 1 -- 172.21.15.5:0/3382907772 shutdown_connections 2023-11-24T06:17:56.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.050+0000 7f31beab4640 1 -- 172.21.15.5:0/3382907772 wait complete. 2023-11-24T06:17:56.071 INFO:tasks.ceph:Scrubbing osd.4 2023-11-24T06:17:56.072 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:17:56.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.303+0000 7ff7eaf83640 1 Processor -- start 2023-11-24T06:17:56.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.303+0000 7ff7eaf83640 1 -- start start 2023-11-24T06:17:56.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.305+0000 7ff7eaf83640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff7e4160bd0 0x7ff7e40b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:56.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.305+0000 7ff7eaf83640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7e40b3210 0x7ff7e4167290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:56.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.305+0000 7ff7eaf83640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e40b23e0 con 0x7ff7e40b48d0 2023-11-24T06:17:56.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.305+0000 7ff7eaf83640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e40cdc70 con 0x7ff7e4160bd0 2023-11-24T06:17:56.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.305+0000 7ff7eaf83640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e40b2870 con 0x7ff7e40b3210 2023-11-24T06:17:56.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.305+0000 7ff7e8d1f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff7e4160bd0 0x7ff7e40b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:56.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.305+0000 7ff7e8d1f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff7e4160bd0 0x7ff7e40b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58960/0 (socket says 172.21.15.5:58960) 2023-11-24T06:17:56.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.305+0000 7ff7e8d1f640 1 -- 172.21.15.5:0/964508367 learned_addr learned my addr 172.21.15.5:0/964508367 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:56.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.306+0000 7ff7e9520640 1 --2- 172.21.15.5:0/964508367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7e40b3210 0x7ff7e4167290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:56.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.306+0000 7ff7e8d1f640 1 -- 172.21.15.5:0/964508367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7e40b3210 msgr2=0x7ff7e4167290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:56.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.306+0000 7ff7e8d1f640 1 --2- 172.21.15.5:0/964508367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7e40b3210 0x7ff7e4167290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.306+0000 7ff7e8d1f640 1 -- 172.21.15.5:0/964508367 >> v1:172.21.15.6:6789/0 conn(0x7ff7e40b48d0 legacy=0x7ff7e40b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:56.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.306+0000 7ff7e8d1f640 1 -- 172.21.15.5:0/964508367 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e40d41b0 con 0x7ff7e4160bd0 2023-11-24T06:17:56.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.307+0000 7ff7e8d1f640 1 --2- 172.21.15.5:0/964508367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e4160bd0 0x7ff7e40b1ea0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009e50 tx=0x7ff7d402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86cab472b1287a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:56.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.307+0000 7ff7e37fe640 1 -- 172.21.15.5:0/964508367 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d402c9e0 con 0x7ff7e4160bd0 2023-11-24T06:17:56.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.307+0000 7ff7e37fe640 1 -- 172.21.15.5:0/964508367 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d402cb40 con 0x7ff7e4160bd0 2023-11-24T06:17:56.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.307+0000 7ff7e37fe640 1 -- 172.21.15.5:0/964508367 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40343f0 con 0x7ff7e4160bd0 2023-11-24T06:17:56.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.308+0000 7ff7eaf83640 1 -- 172.21.15.5:0/964508367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e4160bd0 msgr2=0x7ff7e40b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:56.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.308+0000 7ff7eaf83640 1 --2- 172.21.15.5:0/964508367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e4160bd0 0x7ff7e40b1ea0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009e50 tx=0x7ff7d402be40 comp rx=0 tx=0).stop 2023-11-24T06:17:56.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.308+0000 7ff7eaf83640 1 -- 172.21.15.5:0/964508367 shutdown_connections 2023-11-24T06:17:56.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.308+0000 7ff7eaf83640 1 --2- 172.21.15.5:0/964508367 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7e40b3210 0x7ff7e4167290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.308+0000 7ff7eaf83640 1 --2- 172.21.15.5:0/964508367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e4160bd0 0x7ff7e40b1ea0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.308+0000 7ff7eaf83640 1 -- 172.21.15.5:0/964508367 >> 172.21.15.5:0/964508367 conn(0x7ff7e409c5a0 msgr2=0x7ff7e40d3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:56.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.308+0000 7ff7eaf83640 1 -- 172.21.15.5:0/964508367 shutdown_connections 2023-11-24T06:17:56.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.308+0000 7ff7eaf83640 1 -- 172.21.15.5:0/964508367 wait complete. 2023-11-24T06:17:56.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.310+0000 7ff7eaf83640 1 Processor -- start 2023-11-24T06:17:56.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.310+0000 7ff7eaf83640 1 -- start start 2023-11-24T06:17:56.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.311+0000 7ff7eaf83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e40b3210 0x7ff7e4157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:56.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.311+0000 7ff7eaf83640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7e40b48d0 0x7ff7e40cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:56.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.311+0000 7ff7eaf83640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7e4160bd0 0x7ff7e40b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:56.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.311+0000 7ff7eaf83640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e40ca660 con 0x7ff7e40b3210 2023-11-24T06:17:56.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.311+0000 7ff7eaf83640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e40ca940 con 0x7ff7e40b48d0 2023-11-24T06:17:56.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.311+0000 7ff7eaf83640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e40cac20 con 0x7ff7e4160bd0 2023-11-24T06:17:56.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.311+0000 7ff7e8d1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e40b3210 0x7ff7e4157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:56.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.311+0000 7ff7e8d1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e40b3210 0x7ff7e4157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58962/0 (socket says 172.21.15.5:58962) 2023-11-24T06:17:56.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.311+0000 7ff7e8d1f640 1 -- 172.21.15.5:0/649723563 learned_addr learned my addr 172.21.15.5:0/649723563 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:56.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.312+0000 7ff7e8d1f640 1 -- 172.21.15.5:0/649723563 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7e4160bd0 msgr2=0x7ff7e40b79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:56.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.312+0000 7ff7e8d1f640 1 --2- 172.21.15.5:0/649723563 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7ff7e4160bd0 0x7ff7e40b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.312+0000 7ff7e8d1f640 1 -- 172.21.15.5:0/649723563 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7e40b48d0 msgr2=0x7ff7e40cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:56.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.312+0000 7ff7e8d1f640 1 --2- 172.21.15.5:0/649723563 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7ff7e40b48d0 0x7ff7e40cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.312+0000 7ff7e8d1f640 1 -- 172.21.15.5:0/649723563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4009b30 con 0x7ff7e40b3210 2023-11-24T06:17:56.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.312+0000 7ff7e8d1f640 1 --2- 172.21.15.5:0/649723563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e40b3210 0x7ff7e4157b40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009e50 tx=0x7ff7d4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:56.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.312+0000 7ff7e17fa640 1 -- 172.21.15.5:0/649723563 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d403e450 con 0x7ff7e40b3210 2023-11-24T06:17:56.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.312+0000 7ff7e17fa640 1 -- 172.21.15.5:0/649723563 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d403e5f0 con 0x7ff7e40b3210 2023-11-24T06:17:56.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.312+0000 7ff7eaf83640 1 -- 172.21.15.5:0/649723563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e40caf00 con 0x7ff7e40b3210 2023-11-24T06:17:56.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.312+0000 7ff7e17fa640 1 -- 172.21.15.5:0/649723563 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d403e9a0 con 0x7ff7e40b3210 2023-11-24T06:17:56.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.312+0000 7ff7eaf83640 1 -- 172.21.15.5:0/649723563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e40ca310 con 0x7ff7e40b3210 2023-11-24T06:17:56.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.313+0000 7ff7eaf83640 1 -- 172.21.15.5:0/649723563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff7e40caab0 con 0x7ff7e40b3210 2023-11-24T06:17:56.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.315+0000 7ff7e17fa640 1 -- 172.21.15.5:0/649723563 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ff7d403eb00 con 0x7ff7e40b3210 2023-11-24T06:17:56.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.316+0000 7ff7e17fa640 1 --2- 172.21.15.5:0/649723563 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff7c406aed0 0x7ff7c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:56.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.316+0000 7ff7e17fa640 1 -- 172.21.15.5:0/649723563 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff7d40ac3a0 con 0x7ff7e40b3210 2023-11-24T06:17:56.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.316+0000 7ff7e3fff640 1 --2- 172.21.15.5:0/649723563 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff7c406aed0 0x7ff7c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:56.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.316+0000 7ff7e17fa640 1 --2- 172.21.15.5:0/649723563 >> [v2:172.21.15.6:6822/2056352444,v1:172.21.15.6:6823/2056352444] conn(0x7ff7c4070d30 0x7ff7c4073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:56.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.316+0000 7ff7e17fa640 1 -- 172.21.15.5:0/649723563 --> [v2:172.21.15.6:6822/2056352444,v1:172.21.15.6:6823/2056352444] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7c4073800 con 0x7ff7c4070d30 2023-11-24T06:17:56.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.316+0000 7ff7e17fa640 1 -- 172.21.15.5:0/649723563 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7d40ae980 con 0x7ff7e40b3210 2023-11-24T06:17:56.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.317+0000 7ff7e3fff640 1 --2- 172.21.15.5:0/649723563 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff7c406aed0 0x7ff7c406d390 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4158e30 tx=0x7ff7cc0079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:56.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.317+0000 7ff7e9520640 1 --2- 172.21.15.5:0/649723563 >> [v2:172.21.15.6:6822/2056352444,v1:172.21.15.6:6823/2056352444] conn(0x7ff7c4070d30 0x7ff7c4073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:56.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.318+0000 7ff7e9520640 1 --2- 172.21.15.5:0/649723563 >> [v2:172.21.15.6:6822/2056352444,v1:172.21.15.6:6823/2056352444] conn(0x7ff7c4070d30 0x7ff7c4073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:56.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.320+0000 7ff7e17fa640 1 -- 172.21.15.5:0/649723563 <== osd.4 v2:172.21.15.6:6822/2056352444 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff7c4073800 con 0x7ff7c4070d30 2023-11-24T06:17:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.335+0000 7ff7eaf83640 1 -- 172.21.15.5:0/649723563 --> [v2:172.21.15.6:6822/2056352444,v1:172.21.15.6:6823/2056352444] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff7e40b3760 con 0x7ff7c4070d30 2023-11-24T06:17:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.336+0000 7ff7e17fa640 1 -- 172.21.15.5:0/649723563 <== osd.4 v2:172.21.15.6:6822/2056352444 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff7dc004040 con 0x7ff7c4070d30 2023-11-24T06:17:56.338 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-11-24T06:17:56.338 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:17:56.338 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-11-24T06:17:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.336+0000 7ff7eaf83640 1 -- 172.21.15.5:0/649723563 >> [v2:172.21.15.6:6822/2056352444,v1:172.21.15.6:6823/2056352444] conn(0x7ff7c4070d30 msgr2=0x7ff7c4073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.336+0000 7ff7eaf83640 1 --2- 172.21.15.5:0/649723563 >> [v2:172.21.15.6:6822/2056352444,v1:172.21.15.6:6823/2056352444] conn(0x7ff7c4070d30 0x7ff7c4073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.337+0000 7ff7eaf83640 1 -- 172.21.15.5:0/649723563 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff7c406aed0 msgr2=0x7ff7c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.337+0000 7ff7eaf83640 1 --2- 172.21.15.5:0/649723563 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7ff7c406aed0 0x7ff7c406d390 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4158e30 tx=0x7ff7cc0079b0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.337+0000 7ff7eaf83640 1 -- 172.21.15.5:0/649723563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e40b3210 msgr2=0x7ff7e4157b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.337+0000 7ff7eaf83640 1 --2- 172.21.15.5:0/649723563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7e40b3210 0x7ff7e4157b40 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff7d4009e50 tx=0x7ff7d4004a40 comp rx=0 tx=0).stop 2023-11-24T06:17:56.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.337+0000 7ff7e9520640 1 -- 172.21.15.5:0/649723563 reap_dead start 2023-11-24T06:17:56.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.337+0000 7ff7eaf83640 1 -- 172.21.15.5:0/649723563 shutdown_connections 2023-11-24T06:17:56.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.337+0000 7ff7eaf83640 1 -- 172.21.15.5:0/649723563 >> 172.21.15.5:0/649723563 conn(0x7ff7e409c5a0 msgr2=0x7ff7e40cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:56.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.337+0000 7ff7eaf83640 1 -- 172.21.15.5:0/649723563 shutdown_connections 2023-11-24T06:17:56.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.337+0000 7ff7eaf83640 1 -- 172.21.15.5:0/649723563 wait complete. 2023-11-24T06:17:56.360 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-24T06:17:56.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.588+0000 7f0f3a87e640 1 Processor -- start 2023-11-24T06:17:56.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.589+0000 7f0f3a87e640 1 -- start start 2023-11-24T06:17:56.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.590+0000 7f0f3a87e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f0f3415d430 0x7f0f340d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:56.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.590+0000 7f0f3a87e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f340cfaa0 0x7f0f341615e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:56.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.590+0000 7f0f3a87e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0f34161b20 0x7f0f340d1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:56.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.590+0000 7f0f3a87e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f340d09d0 con 0x7f0f340cfaa0 2023-11-24T06:17:56.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.590+0000 7f0f3a87e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f340d1030 con 0x7f0f3415d430 2023-11-24T06:17:56.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.590+0000 7f0f3a87e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f340cff30 con 0x7f0f34161b20 2023-11-24T06:17:56.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.591+0000 7f0f3907b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f340cfaa0 0x7f0f341615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.591+0000 7f0f3907b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0f340cfaa0 0x7f0f341615e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58976/0 (socket says 172.21.15.5:58976) 2023-11-24T06:17:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.591+0000 7f0f3907b640 1 -- 172.21.15.5:0/3283904812 learned_addr learned my addr 172.21.15.5:0/3283904812 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.591+0000 7f0f3a07d640 1 --2- 172.21.15.5:0/3283904812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0f34161b20 0x7f0f340d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.591+0000 7f0f3987c640 1 --2- 172.21.15.5:0/3283904812 >> v2:172.21.15.6:3300/0 conn(0x7f0f3415d430 0x7f0f340d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.591+0000 7f0f3907b640 1 -- 172.21.15.5:0/3283904812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0f34161b20 msgr2=0x7f0f340d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.591+0000 7f0f3907b640 1 --2- 172.21.15.5:0/3283904812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0f34161b20 0x7f0f340d1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.591+0000 7f0f3907b640 1 -- 172.21.15.5:0/3283904812 >> v2:172.21.15.6:3300/0 conn(0x7f0f3415d430 msgr2=0x7f0f340d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.591+0000 7f0f3907b640 1 --2- 172.21.15.5:0/3283904812 >> v2:172.21.15.6:3300/0 conn(0x7f0f3415d430 0x7f0f340d3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.591+0000 7f0f3907b640 1 -- 172.21.15.5:0/3283904812 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3415d810 con 0x7f0f340cfaa0 2023-11-24T06:17:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.593+0000 7f0f3907b640 1 --2- 172.21.15.5:0/3283904812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f340cfaa0 0x7f0f341615e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0f340b2ca0 tx=0x7f0f2402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989c12eb7ace62a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:56.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.593+0000 7f0f3887a640 1 -- 172.21.15.5:0/3283904812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f240317e0 con 0x7f0f340cfaa0 2023-11-24T06:17:56.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.593+0000 7f0f3887a640 1 -- 172.21.15.5:0/3283904812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f24031940 con 0x7f0f340cfaa0 2023-11-24T06:17:56.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.593+0000 7f0f3887a640 1 -- 172.21.15.5:0/3283904812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24031cd0 con 0x7f0f340cfaa0 2023-11-24T06:17:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.594+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3283904812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f340cfaa0 msgr2=0x7f0f341615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.594+0000 7f0f3a87e640 1 --2- 172.21.15.5:0/3283904812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f340cfaa0 0x7f0f341615e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0f340b2ca0 tx=0x7f0f2402b880 comp rx=0 tx=0).stop 2023-11-24T06:17:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.594+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3283904812 shutdown_connections 2023-11-24T06:17:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.594+0000 7f0f3a87e640 1 --2- 172.21.15.5:0/3283904812 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0f34161b20 0x7f0f340d1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.594+0000 7f0f3a87e640 1 --2- 172.21.15.5:0/3283904812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f340cfaa0 0x7f0f341615e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.594+0000 7f0f3a87e640 1 --2- 172.21.15.5:0/3283904812 >> v2:172.21.15.6:3300/0 conn(0x7f0f3415d430 0x7f0f340d3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.594+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3283904812 >> 172.21.15.5:0/3283904812 conn(0x7f0f3408cb00 msgr2=0x7f0f3409c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.594+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3283904812 shutdown_connections 2023-11-24T06:17:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.594+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3283904812 wait complete. 2023-11-24T06:17:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.596+0000 7f0f3a87e640 1 Processor -- start 2023-11-24T06:17:56.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.596+0000 7f0f3a87e640 1 -- start start 2023-11-24T06:17:56.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.597+0000 7f0f3a87e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f340cfaa0 0x7f0f340ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:56.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.597+0000 7f0f3a87e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f3415d430 0x7f0f340cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.597+0000 7f0f3a87e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0f34161b20 0x7f0f340cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.597+0000 7f0f3a87e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3415e010 con 0x7f0f3415d430 2023-11-24T06:17:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.597+0000 7f0f3a87e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f340cccd0 con 0x7f0f340cfaa0 2023-11-24T06:17:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.597+0000 7f0f3a87e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f340ccfb0 con 0x7f0f34161b20 2023-11-24T06:17:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.598+0000 7f0f3907b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f3415d430 0x7f0f340cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.598+0000 7f0f3907b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f3415d430 0x7f0f340cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58978/0 (socket says 172.21.15.5:58978) 2023-11-24T06:17:56.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.598+0000 7f0f3907b640 1 -- 172.21.15.5:0/3235524988 learned_addr learned my addr 172.21.15.5:0/3235524988 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:56.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.598+0000 7f0f3987c640 1 --2- 172.21.15.5:0/3235524988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f340cfaa0 0x7f0f340ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.598+0000 7f0f3907b640 1 -- 172.21.15.5:0/3235524988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0f34161b20 msgr2=0x7f0f340cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.598+0000 7f0f3907b640 1 --2- 172.21.15.5:0/3235524988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0f34161b20 0x7f0f340cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.598+0000 7f0f3907b640 1 -- 172.21.15.5:0/3235524988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f340cfaa0 msgr2=0x7f0f340ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.598+0000 7f0f3907b640 1 --2- 172.21.15.5:0/3235524988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f340cfaa0 0x7f0f340ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.598+0000 7f0f3907b640 1 -- 172.21.15.5:0/3235524988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f240094d0 con 0x7f0f3415d430 2023-11-24T06:17:56.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.599+0000 7f0f3907b640 1 --2- 172.21.15.5:0/3235524988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f3415d430 0x7f0f340cc130 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0f240090b0 tx=0x7f0f240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.599+0000 7f0f2a7fc640 1 -- 172.21.15.5:0/3235524988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24032880 con 0x7f0f3415d430 2023-11-24T06:17:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.599+0000 7f0f2a7fc640 1 -- 172.21.15.5:0/3235524988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f240329e0 con 0x7f0f3415d430 2023-11-24T06:17:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.599+0000 7f0f2a7fc640 1 -- 172.21.15.5:0/3235524988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24032d90 con 0x7f0f3415d430 2023-11-24T06:17:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.599+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3235524988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3415e2d0 con 0x7f0f3415d430 2023-11-24T06:17:56.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.599+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3235524988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3414d4b0 con 0x7f0f3415d430 2023-11-24T06:17:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.600+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3235524988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f34067960 con 0x7f0f3415d430 2023-11-24T06:17:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.601+0000 7f0f2a7fc640 1 -- 172.21.15.5:0/3235524988 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0f2404e450 con 0x7f0f3415d430 2023-11-24T06:17:56.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.602+0000 7f0f2a7fc640 1 --2- 172.21.15.5:0/3235524988 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0f1806aed0 0x7f0f1806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:56.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.602+0000 7f0f2a7fc640 1 -- 172.21.15.5:0/3235524988 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0f240ab670 con 0x7f0f3415d430 2023-11-24T06:17:56.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.602+0000 7f0f3987c640 1 --2- 172.21.15.5:0/3235524988 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0f1806aed0 0x7f0f1806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:56.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.603+0000 7f0f3987c640 1 --2- 172.21.15.5:0/3235524988 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0f1806aed0 0x7f0f1806d390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0f30006fd0 tx=0x7f0f30008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:56.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.607+0000 7f0f2a7fc640 1 -- 172.21.15.5:0/3235524988 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f240768d0 con 0x7f0f3415d430 2023-11-24T06:17:56.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.803+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3235524988 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f3415e150 con 0x7f0f1806aed0 2023-11-24T06:17:56.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.804+0000 7f0f2a7fc640 1 -- 172.21.15.5:0/3235524988 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0f3000e510 con 0x7f0f1806aed0 2023-11-24T06:17:56.806 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 4 to deep-scrub 2023-11-24T06:17:56.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.806+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3235524988 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0f1806aed0 msgr2=0x7f0f1806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:56.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.806+0000 7f0f3a87e640 1 --2- 172.21.15.5:0/3235524988 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0f1806aed0 0x7f0f1806d390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0f30006fd0 tx=0x7f0f30008040 comp rx=0 tx=0).stop 2023-11-24T06:17:56.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.806+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3235524988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f3415d430 msgr2=0x7f0f340cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:56.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.806+0000 7f0f3a87e640 1 --2- 172.21.15.5:0/3235524988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f3415d430 0x7f0f340cc130 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0f240090b0 tx=0x7f0f240046c0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.806+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3235524988 shutdown_connections 2023-11-24T06:17:56.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.806+0000 7f0f3a87e640 1 --2- 172.21.15.5:0/3235524988 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f0f1806aed0 0x7f0f1806d390 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.806+0000 7f0f3a87e640 1 --2- 172.21.15.5:0/3235524988 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f0f34161b20 0x7f0f340cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.806+0000 7f0f3a87e640 1 --2- 172.21.15.5:0/3235524988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f3415d430 0x7f0f340cc130 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.806+0000 7f0f3a87e640 1 --2- 172.21.15.5:0/3235524988 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f0f340cfaa0 0x7f0f340ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:56.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.806+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3235524988 >> 172.21.15.5:0/3235524988 conn(0x7f0f3408cb00 msgr2=0x7f0f340d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:56.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.807+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3235524988 shutdown_connections 2023-11-24T06:17:56.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:56.807+0000 7f0f3a87e640 1 -- 172.21.15.5:0/3235524988 wait complete. 2023-11-24T06:17:56.829 INFO:tasks.ceph:Scrubbing osd.5 2023-11-24T06:17:56.829 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:17:57.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.060+0000 7fe28d5e6640 1 Processor -- start 2023-11-24T06:17:57.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.060+0000 7fe28d5e6640 1 -- start start 2023-11-24T06:17:57.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.062+0000 7fe28d5e6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe2880ad450 0x7fe288154280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.062+0000 7fe28d5e6640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe288070850 0x7fe288071350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.062+0000 7fe28d5e6640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2880ce650 con 0x7fe288153ea0 2023-11-24T06:17:57.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.062+0000 7fe28d5e6640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2880ad8e0 con 0x7fe2880ad450 2023-11-24T06:17:57.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.063+0000 7fe28d5e6640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2881547c0 con 0x7fe288070850 2023-11-24T06:17:57.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.063+0000 7fe287fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe288153ea0 0x7fe28815d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50674/0 (socket says 172.21.15.5:50674) 2023-11-24T06:17:57.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.063+0000 7fe287fff640 1 -- 172.21.15.5:0/2138880342 learned_addr learned my addr 172.21.15.5:0/2138880342 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:57.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.063+0000 7fe2877fe640 1 --2- 172.21.15.5:0/2138880342 >> v2:172.21.15.5:3300/0 conn(0x7fe2880ad450 0x7fe288154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.063+0000 7fe28cde5640 1 --2- 172.21.15.5:0/2138880342 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe288070850 0x7fe288071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.063+0000 7fe2877fe640 1 -- 172.21.15.5:0/2138880342 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe288070850 msgr2=0x7fe288071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.063+0000 7fe2877fe640 1 --2- 172.21.15.5:0/2138880342 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe288070850 0x7fe288071350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.063+0000 7fe2877fe640 1 -- 172.21.15.5:0/2138880342 >> v1:172.21.15.5:6789/0 conn(0x7fe288153ea0 legacy=0x7fe28815d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.063+0000 7fe2877fe640 1 -- 172.21.15.5:0/2138880342 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28815e0a0 con 0x7fe2880ad450 2023-11-24T06:17:57.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.065+0000 7fe2877fe640 1 --2- 172.21.15.5:0/2138880342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2880ad450 0x7fe288154280 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0097d0 tx=0x7fe27c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6789c79feefc46b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.065+0000 7fe286ffd640 1 -- 172.21.15.5:0/2138880342 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c031800 con 0x7fe2880ad450 2023-11-24T06:17:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.065+0000 7fe286ffd640 1 -- 172.21.15.5:0/2138880342 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c031960 con 0x7fe2880ad450 2023-11-24T06:17:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.065+0000 7fe286ffd640 1 -- 172.21.15.5:0/2138880342 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c031cf0 con 0x7fe2880ad450 2023-11-24T06:17:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.066+0000 7fe28d5e6640 1 -- 172.21.15.5:0/2138880342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2880ad450 msgr2=0x7fe288154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.066+0000 7fe28d5e6640 1 --2- 172.21.15.5:0/2138880342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2880ad450 0x7fe288154280 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0097d0 tx=0x7fe27c02b8a0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.066+0000 7fe28d5e6640 1 -- 172.21.15.5:0/2138880342 shutdown_connections 2023-11-24T06:17:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.066+0000 7fe28d5e6640 1 --2- 172.21.15.5:0/2138880342 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe288070850 0x7fe288071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.066+0000 7fe28d5e6640 1 --2- 172.21.15.5:0/2138880342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2880ad450 0x7fe288154280 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.066+0000 7fe28d5e6640 1 -- 172.21.15.5:0/2138880342 >> 172.21.15.5:0/2138880342 conn(0x7fe2880cec10 msgr2=0x7fe2880cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.066+0000 7fe28d5e6640 1 -- 172.21.15.5:0/2138880342 shutdown_connections 2023-11-24T06:17:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.066+0000 7fe28d5e6640 1 -- 172.21.15.5:0/2138880342 wait complete. 2023-11-24T06:17:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.068+0000 7fe28d5e6640 1 Processor -- start 2023-11-24T06:17:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.068+0000 7fe28d5e6640 1 -- start start 2023-11-24T06:17:57.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.069+0000 7fe28d5e6640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe288070850 0x7fe2881395e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.069+0000 7fe28d5e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2880ad450 0x7fe28814a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.069+0000 7fe28d5e6640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe288153ea0 0x7fe288157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.069+0000 7fe28d5e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe288156230 con 0x7fe2880ad450 2023-11-24T06:17:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.069+0000 7fe28d5e6640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe288156db0 con 0x7fe288153ea0 2023-11-24T06:17:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.069+0000 7fe28d5e6640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe288139b20 con 0x7fe288070850 2023-11-24T06:17:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.069+0000 7fe2877fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2880ad450 0x7fe28814a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.069+0000 7fe2877fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2880ad450 0x7fe28814a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59004/0 (socket says 172.21.15.5:59004) 2023-11-24T06:17:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.069+0000 7fe2877fe640 1 -- 172.21.15.5:0/3001064684 learned_addr learned my addr 172.21.15.5:0/3001064684 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.070+0000 7fe2877fe640 1 -- 172.21.15.5:0/3001064684 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe288070850 msgr2=0x7fe2881395e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.070+0000 7fe2877fe640 1 --2- 172.21.15.5:0/3001064684 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe288070850 0x7fe2881395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.070+0000 7fe2877fe640 1 -- 172.21.15.5:0/3001064684 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe288153ea0 msgr2=0x7fe288157240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.070+0000 7fe2877fe640 1 --2- 172.21.15.5:0/3001064684 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe288153ea0 0x7fe288157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.070+0000 7fe2877fe640 1 -- 172.21.15.5:0/3001064684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c009400 con 0x7fe2880ad450 2023-11-24T06:17:57.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.070+0000 7fe2877fe640 1 --2- 172.21.15.5:0/3001064684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2880ad450 0x7fe28814a4e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe27c02beb0 tx=0x7fe27c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:57.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.070+0000 7fe284ff9640 1 -- 172.21.15.5:0/3001064684 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c03bd30 con 0x7fe2880ad450 2023-11-24T06:17:57.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.071+0000 7fe284ff9640 1 -- 172.21.15.5:0/3001064684 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c03be90 con 0x7fe2880ad450 2023-11-24T06:17:57.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.071+0000 7fe284ff9640 1 -- 172.21.15.5:0/3001064684 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c045630 con 0x7fe2880ad450 2023-11-24T06:17:57.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.071+0000 7fe28d5e6640 1 -- 172.21.15.5:0/3001064684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe288156480 con 0x7fe2880ad450 2023-11-24T06:17:57.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.071+0000 7fe28d5e6640 1 -- 172.21.15.5:0/3001064684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe288156a40 con 0x7fe2880ad450 2023-11-24T06:17:57.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.072+0000 7fe28d5e6640 1 -- 172.21.15.5:0/3001064684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe288070050 con 0x7fe2880ad450 2023-11-24T06:17:57.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.073+0000 7fe284ff9640 1 -- 172.21.15.5:0/3001064684 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe27c04e450 con 0x7fe2880ad450 2023-11-24T06:17:57.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.074+0000 7fe284ff9640 1 --2- 172.21.15.5:0/3001064684 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe27406af00 0x7fe27406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.074+0000 7fe284ff9640 1 -- 172.21.15.5:0/3001064684 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe27c02e070 con 0x7fe2880ad450 2023-11-24T06:17:57.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.074+0000 7fe287fff640 1 --2- 172.21.15.5:0/3001064684 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe27406af00 0x7fe27406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.074+0000 7fe284ff9640 1 --2- 172.21.15.5:0/3001064684 >> [v2:172.21.15.6:6800/3179572934,v1:172.21.15.6:6801/3179572934] conn(0x7fe274070d60 0x7fe274073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.074+0000 7fe284ff9640 1 -- 172.21.15.5:0/3001064684 --> [v2:172.21.15.6:6800/3179572934,v1:172.21.15.6:6801/3179572934] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe274073830 con 0x7fe274070d60 2023-11-24T06:17:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.074+0000 7fe284ff9640 1 -- 172.21.15.5:0/3001064684 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe27c0adc80 con 0x7fe2880ad450 2023-11-24T06:17:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.075+0000 7fe287fff640 1 --2- 172.21.15.5:0/3001064684 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe27406af00 0x7fe27406d3c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe278001550 tx=0x7fe278007a40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.075+0000 7fe28cde5640 1 --2- 172.21.15.5:0/3001064684 >> [v2:172.21.15.6:6800/3179572934,v1:172.21.15.6:6801/3179572934] conn(0x7fe274070d60 0x7fe274073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.076+0000 7fe28cde5640 1 --2- 172.21.15.5:0/3001064684 >> [v2:172.21.15.6:6800/3179572934,v1:172.21.15.6:6801/3179572934] conn(0x7fe274070d60 0x7fe274073180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:57.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.078+0000 7fe284ff9640 1 -- 172.21.15.5:0/3001064684 <== osd.5 v2:172.21.15.6:6800/3179572934 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe274073830 con 0x7fe274070d60 2023-11-24T06:17:57.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.092+0000 7fe28d5e6640 1 -- 172.21.15.5:0/3001064684 --> [v2:172.21.15.6:6800/3179572934,v1:172.21.15.6:6801/3179572934] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe28815e660 con 0x7fe274070d60 2023-11-24T06:17:57.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.093+0000 7fe284ff9640 1 -- 172.21.15.5:0/3001064684 <== osd.5 v2:172.21.15.6:6800/3179572934 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe280009ad0 con 0x7fe274070d60 2023-11-24T06:17:57.095 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-11-24T06:17:57.096 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:17:57.096 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-11-24T06:17:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.094+0000 7fe28d5e6640 1 -- 172.21.15.5:0/3001064684 >> [v2:172.21.15.6:6800/3179572934,v1:172.21.15.6:6801/3179572934] conn(0x7fe274070d60 msgr2=0x7fe274073180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.094+0000 7fe28d5e6640 1 --2- 172.21.15.5:0/3001064684 >> [v2:172.21.15.6:6800/3179572934,v1:172.21.15.6:6801/3179572934] conn(0x7fe274070d60 0x7fe274073180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.094+0000 7fe28d5e6640 1 -- 172.21.15.5:0/3001064684 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe27406af00 msgr2=0x7fe27406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.094+0000 7fe28d5e6640 1 --2- 172.21.15.5:0/3001064684 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe27406af00 0x7fe27406d3c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe278001550 tx=0x7fe278007a40 comp rx=0 tx=0).stop 2023-11-24T06:17:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.094+0000 7fe28d5e6640 1 -- 172.21.15.5:0/3001064684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2880ad450 msgr2=0x7fe28814a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.094+0000 7fe28d5e6640 1 --2- 172.21.15.5:0/3001064684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2880ad450 0x7fe28814a4e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe27c02beb0 tx=0x7fe27c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.094+0000 7fe28cde5640 1 -- 172.21.15.5:0/3001064684 reap_dead start 2023-11-24T06:17:57.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.094+0000 7fe28d5e6640 1 -- 172.21.15.5:0/3001064684 shutdown_connections 2023-11-24T06:17:57.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.094+0000 7fe28d5e6640 1 -- 172.21.15.5:0/3001064684 >> 172.21.15.5:0/3001064684 conn(0x7fe2880cec10 msgr2=0x7fe28806c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:57.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.094+0000 7fe28d5e6640 1 -- 172.21.15.5:0/3001064684 shutdown_connections 2023-11-24T06:17:57.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.094+0000 7fe28d5e6640 1 -- 172.21.15.5:0/3001064684 wait complete. 2023-11-24T06:17:57.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-24T06:17:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.329+0000 7fe39f59e640 1 Processor -- start 2023-11-24T06:17:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.329+0000 7fe39f59e640 1 -- start start 2023-11-24T06:17:57.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.330+0000 7fe39f59e640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fe3a0070300 0x7fe3a0070740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.330+0000 7fe39f59e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3a0148e70 0x7fe3a0149300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.330+0000 7fe39f59e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a01489e0 con 0x7fe3a0147f50 2023-11-24T06:17:57.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.330+0000 7fe39f59e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a0070c80 con 0x7fe3a0148e70 2023-11-24T06:17:57.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.330+0000 7fe39f59e640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a006d230 con 0x7fe3a0070300 2023-11-24T06:17:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.331+0000 7fe39ed9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3a0148e70 0x7fe3a0149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.331+0000 7fe39e59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe3a0147f50 0x7fe3a006cb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50690/0 (socket says 172.21.15.5:50690) 2023-11-24T06:17:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.331+0000 7fe39e59c640 1 -- 172.21.15.5:0/1977642447 learned_addr learned my addr 172.21.15.5:0/1977642447 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.331+0000 7fe39dd9b640 1 --2- 172.21.15.5:0/1977642447 >> v2:172.21.15.6:3300/0 conn(0x7fe3a0070300 0x7fe3a0070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.331+0000 7fe39ed9d640 1 -- 172.21.15.5:0/1977642447 >> v2:172.21.15.6:3300/0 conn(0x7fe3a0070300 msgr2=0x7fe3a0070740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.331+0000 7fe39ed9d640 1 --2- 172.21.15.5:0/1977642447 >> v2:172.21.15.6:3300/0 conn(0x7fe3a0070300 0x7fe3a0070740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.331+0000 7fe39ed9d640 1 -- 172.21.15.5:0/1977642447 >> v1:172.21.15.5:6789/0 conn(0x7fe3a0147f50 legacy=0x7fe3a006cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.331+0000 7fe39ed9d640 1 -- 172.21.15.5:0/1977642447 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a0148330 con 0x7fe3a0148e70 2023-11-24T06:17:57.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.331+0000 7fe39d59a640 1 -- 172.21.15.5:0/1977642447 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861781888 0 0) 0x7fe3a01489e0 con 0x7fe3a0147f50 2023-11-24T06:17:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.333+0000 7fe39ed9d640 1 --2- 172.21.15.5:0/1977642447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a0148e70 0x7fe3a0149300 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe3a00b18d0 tx=0x7fe39802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb60c4e60ebe600f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.333+0000 7fe39d59a640 1 -- 172.21.15.5:0/1977642447 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398030610 con 0x7fe3a0148e70 2023-11-24T06:17:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.333+0000 7fe39d59a640 1 -- 172.21.15.5:0/1977642447 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe398030770 con 0x7fe3a0148e70 2023-11-24T06:17:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.333+0000 7fe39d59a640 1 -- 172.21.15.5:0/1977642447 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398030b00 con 0x7fe3a0148e70 2023-11-24T06:17:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.333+0000 7fe39f59e640 1 -- 172.21.15.5:0/1977642447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a0148e70 msgr2=0x7fe3a0149300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.333+0000 7fe39f59e640 1 --2- 172.21.15.5:0/1977642447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a0148e70 0x7fe3a0149300 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe3a00b18d0 tx=0x7fe39802b960 comp rx=0 tx=0).stop 2023-11-24T06:17:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.334+0000 7fe39f59e640 1 -- 172.21.15.5:0/1977642447 shutdown_connections 2023-11-24T06:17:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.334+0000 7fe39f59e640 1 --2- 172.21.15.5:0/1977642447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a0148e70 0x7fe3a0149300 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.334+0000 7fe39f59e640 1 --2- 172.21.15.5:0/1977642447 >> v2:172.21.15.6:3300/0 conn(0x7fe3a0070300 0x7fe3a0070740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.334+0000 7fe39f59e640 1 -- 172.21.15.5:0/1977642447 >> 172.21.15.5:0/1977642447 conn(0x7fe3a00b0840 msgr2=0x7fe3a00b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.334+0000 7fe39f59e640 1 -- 172.21.15.5:0/1977642447 shutdown_connections 2023-11-24T06:17:57.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.334+0000 7fe39f59e640 1 -- 172.21.15.5:0/1977642447 wait complete. 2023-11-24T06:17:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.335+0000 7fe39f59e640 1 Processor -- start 2023-11-24T06:17:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.336+0000 7fe39f59e640 1 -- start start 2023-11-24T06:17:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.336+0000 7fe39f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a0070300 0x7fe3a0146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.336+0000 7fe39f59e640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3a0147f50 0x7fe3a00cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.336+0000 7fe39f59e640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3a0148e70 0x7fe3a01f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39f59e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00cc800 con 0x7fe3a0070300 2023-11-24T06:17:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39f59e640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00ccae0 con 0x7fe3a0148e70 2023-11-24T06:17:57.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39f59e640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00ccdc0 con 0x7fe3a0147f50 2023-11-24T06:17:57.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a0070300 0x7fe3a0146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a0070300 0x7fe3a0146f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59018/0 (socket says 172.21.15.5:59018) 2023-11-24T06:17:57.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39e59c640 1 -- 172.21.15.5:0/1805600559 learned_addr learned my addr 172.21.15.5:0/1805600559 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:57.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39ed9d640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3a0148e70 0x7fe3a01f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39dd9b640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3a0147f50 0x7fe3a00cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39e59c640 1 -- 172.21.15.5:0/1805600559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3a0147f50 msgr2=0x7fe3a00cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39e59c640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3a0147f50 0x7fe3a00cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39e59c640 1 -- 172.21.15.5:0/1805600559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3a0148e70 msgr2=0x7fe3a01f6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39e59c640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3a0148e70 0x7fe3a01f6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39e59c640 1 -- 172.21.15.5:0/1805600559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe398009400 con 0x7fe3a0070300 2023-11-24T06:17:57.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39ed9d640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3a0148e70 0x7fe3a01f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:57.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.337+0000 7fe39dd9b640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3a0147f50 0x7fe3a00cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:17:57.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.338+0000 7fe39e59c640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a0070300 0x7fe3a0146f40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe39000c4d0 tx=0x7fe39000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:57.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.338+0000 7fe396ffd640 1 -- 172.21.15.5:0/1805600559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3900074e0 con 0x7fe3a0070300 2023-11-24T06:17:57.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.338+0000 7fe396ffd640 1 -- 172.21.15.5:0/1805600559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe390007660 con 0x7fe3a0070300 2023-11-24T06:17:57.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.338+0000 7fe396ffd640 1 -- 172.21.15.5:0/1805600559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390004a20 con 0x7fe3a0070300 2023-11-24T06:17:57.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.338+0000 7fe39f59e640 1 -- 172.21.15.5:0/1805600559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a00cd0a0 con 0x7fe3a0070300 2023-11-24T06:17:57.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.338+0000 7fe39f59e640 1 -- 172.21.15.5:0/1805600559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a00cc490 con 0x7fe3a0070300 2023-11-24T06:17:57.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.340+0000 7fe39f59e640 1 -- 172.21.15.5:0/1805600559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a00cd5c0 con 0x7fe3a0070300 2023-11-24T06:17:57.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.340+0000 7fe396ffd640 1 -- 172.21.15.5:0/1805600559 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe390004b80 con 0x7fe3a0070300 2023-11-24T06:17:57.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.341+0000 7fe396ffd640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe38006aed0 0x7fe38006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.341+0000 7fe396ffd640 1 -- 172.21.15.5:0/1805600559 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe390088630 con 0x7fe3a0070300 2023-11-24T06:17:57.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.342+0000 7fe39dd9b640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe38006aed0 0x7fe38006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.342+0000 7fe39dd9b640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe38006aed0 0x7fe38006d390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe388007330 tx=0x7fe38800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:57.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.346+0000 7fe396ffd640 1 -- 172.21.15.5:0/1805600559 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3a00cd5c0 con 0x7fe3a0070300 2023-11-24T06:17:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.541+0000 7fe39f59e640 1 -- 172.21.15.5:0/1805600559 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3a00ccc50 con 0x7fe38006aed0 2023-11-24T06:17:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.542+0000 7fe396ffd640 1 -- 172.21.15.5:0/1805600559 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe3a00ccc50 con 0x7fe38006aed0 2023-11-24T06:17:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 5 to deep-scrub 2023-11-24T06:17:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.544+0000 7fe39f59e640 1 -- 172.21.15.5:0/1805600559 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe38006aed0 msgr2=0x7fe38006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.544+0000 7fe39f59e640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe38006aed0 0x7fe38006d390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe388007330 tx=0x7fe38800b040 comp rx=0 tx=0).stop 2023-11-24T06:17:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.544+0000 7fe39f59e640 1 -- 172.21.15.5:0/1805600559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a0070300 msgr2=0x7fe3a0146f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.544+0000 7fe39f59e640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a0070300 0x7fe3a0146f40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe39000c4d0 tx=0x7fe39000c990 comp rx=0 tx=0).stop 2023-11-24T06:17:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.545+0000 7fe39f59e640 1 -- 172.21.15.5:0/1805600559 shutdown_connections 2023-11-24T06:17:57.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.545+0000 7fe39f59e640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe38006aed0 0x7fe38006d390 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.545+0000 7fe39f59e640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe3a0147f50 0x7fe3a00cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.545+0000 7fe39f59e640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe3a0148e70 0x7fe3a01f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.545+0000 7fe39f59e640 1 --2- 172.21.15.5:0/1805600559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3a0070300 0x7fe3a0146f40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.545+0000 7fe39f59e640 1 -- 172.21.15.5:0/1805600559 >> 172.21.15.5:0/1805600559 conn(0x7fe3a00b0840 msgr2=0x7fe3a01638a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:57.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.545+0000 7fe39f59e640 1 -- 172.21.15.5:0/1805600559 shutdown_connections 2023-11-24T06:17:57.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.545+0000 7fe39f59e640 1 -- 172.21.15.5:0/1805600559 wait complete. 2023-11-24T06:17:57.567 INFO:tasks.ceph:Scrubbing osd.6 2023-11-24T06:17:57.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:17:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.793+0000 7fa1cb25d640 1 Processor -- start 2023-11-24T06:17:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.793+0000 7fa1cb25d640 1 -- start start 2023-11-24T06:17:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.795+0000 7fa1cb25d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c40ae270 0x7fa1c40abe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.795+0000 7fa1cb25d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c415b730 con 0x7fa1c409cb30 2023-11-24T06:17:57.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.795+0000 7fa1cb25d640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c40ac740 con 0x7fa1c415b350 2023-11-24T06:17:57.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.795+0000 7fa1cb25d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c40ac460 con 0x7fa1c40ae270 2023-11-24T06:17:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.795+0000 7fa1c3fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa1c409cb30 0x7fa1c40adb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50700/0 (socket says 172.21.15.5:50700) 2023-11-24T06:17:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.795+0000 7fa1c3fff640 1 -- 172.21.15.5:0/3326710523 learned_addr learned my addr 172.21.15.5:0/3326710523 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.795+0000 7fa1c97fa640 1 --2- 172.21.15.5:0/3326710523 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c40ae270 0x7fa1c40abe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 -- 172.21.15.5:0/3326710523 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584419065 0 0) 0x7fa1b0001110 con 0x7fa1c409cb30 2023-11-24T06:17:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 -- 172.21.15.5:0/3326710523 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1b4003520 con 0x7fa1c409cb30 2023-11-24T06:17:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 -- 172.21.15.5:0/3326710523 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 77619647 0 0) 0x7fa1b4003520 con 0x7fa1c409cb30 2023-11-24T06:17:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 -- 172.21.15.5:0/3326710523 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c40ae270 msgr2=0x7fa1c40abe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 --2- 172.21.15.5:0/3326710523 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c40ae270 0x7fa1c40abe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 -- 172.21.15.5:0/3326710523 >> v1:172.21.15.6:6789/0 conn(0x7fa1c415b350 legacy=0x7fa1c40b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 -- 172.21.15.5:0/3326710523 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c40b2a50 con 0x7fa1c409cb30 2023-11-24T06:17:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 -- 172.21.15.5:0/3326710523 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7fa1b0001800 con 0x7fa1c409cb30 2023-11-24T06:17:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 -- 172.21.15.5:0/3326710523 >> v1:172.21.15.5:6789/0 conn(0x7fa1c409cb30 legacy=0x7fa1c40adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 --2- 172.21.15.5:0/3326710523 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c40ae270 0x7fa1b40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 --2- 172.21.15.5:0/3326710523 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1b4003f20 0x7fa1b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 --2- 172.21.15.5:0/3326710523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4004d00 0x7fa1b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 -- 172.21.15.5:0/3326710523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c40ac460 con 0x7fa1b4004d00 2023-11-24T06:17:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 -- 172.21.15.5:0/3326710523 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b400ba60 con 0x7fa1b4003f20 2023-11-24T06:17:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c37fe640 1 -- 172.21.15.5:0/3326710523 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b400bc20 con 0x7fa1c40ae270 2023-11-24T06:17:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.796+0000 7fa1c97fa640 1 --2- 172.21.15.5:0/3326710523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4004d00 0x7fa1b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.797+0000 7fa1c8ff9640 1 --2- 172.21.15.5:0/3326710523 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c40ae270 0x7fa1b40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.797+0000 7fa1c3fff640 1 --2- 172.21.15.5:0/3326710523 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1b4003f20 0x7fa1b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.797+0000 7fa1c97fa640 1 -- 172.21.15.5:0/3326710523 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c40ae270 msgr2=0x7fa1b40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.797+0000 7fa1c97fa640 1 --2- 172.21.15.5:0/3326710523 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c40ae270 0x7fa1b40039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.797+0000 7fa1c97fa640 1 -- 172.21.15.5:0/3326710523 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1b4003f20 msgr2=0x7fa1b40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:57.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.797+0000 7fa1c97fa640 1 --2- 172.21.15.5:0/3326710523 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1b4003f20 0x7fa1b40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.797+0000 7fa1c97fa640 1 -- 172.21.15.5:0/3326710523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b4003520 con 0x7fa1b4004d00 2023-11-24T06:17:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.798+0000 7fa1c97fa640 1 --2- 172.21.15.5:0/3326710523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4004d00 0x7fa1b4005180 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009fc0 tx=0x7fa1b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316ee1d2879cb45a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.798+0000 7fa1c37fe640 1 -- 172.21.15.5:0/3326710523 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc02cc20 con 0x7fa1b4004d00 2023-11-24T06:17:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.798+0000 7fa1c37fe640 1 -- 172.21.15.5:0/3326710523 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc00f040 con 0x7fa1b4004d00 2023-11-24T06:17:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.798+0000 7fa1c37fe640 1 -- 172.21.15.5:0/3326710523 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc034440 con 0x7fa1b4004d00 2023-11-24T06:17:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.799+0000 7fa1cb25d640 1 -- 172.21.15.5:0/3326710523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4004d00 msgr2=0x7fa1b4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.799+0000 7fa1cb25d640 1 --2- 172.21.15.5:0/3326710523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4004d00 0x7fa1b4005180 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc009fc0 tx=0x7fa1b40057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.799+0000 7fa1c97fa640 1 -- 172.21.15.5:0/3326710523 reap_dead start 2023-11-24T06:17:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.799+0000 7fa1cb25d640 1 -- 172.21.15.5:0/3326710523 shutdown_connections 2023-11-24T06:17:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.799+0000 7fa1cb25d640 1 -- 172.21.15.5:0/3326710523 >> 172.21.15.5:0/3326710523 conn(0x7fa1c40b0ba0 msgr2=0x7fa1c40b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.799+0000 7fa1cb25d640 1 -- 172.21.15.5:0/3326710523 shutdown_connections 2023-11-24T06:17:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.799+0000 7fa1cb25d640 1 -- 172.21.15.5:0/3326710523 wait complete. 2023-11-24T06:17:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.801+0000 7fa1cb25d640 1 Processor -- start 2023-11-24T06:17:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.801+0000 7fa1cb25d640 1 -- start start 2023-11-24T06:17:57.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.802+0000 7fa1cb25d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4004d00 0x7fa1c4146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.802+0000 7fa1cb25d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c41544d0 0x7fa1c415da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.802+0000 7fa1cb25d640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c415df50 0x7fa1c415ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.802+0000 7fa1cb25d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c4147090 con 0x7fa1b4004d00 2023-11-24T06:17:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.802+0000 7fa1cb25d640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c41472e0 con 0x7fa1c415df50 2023-11-24T06:17:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.802+0000 7fa1cb25d640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c415e3d0 con 0x7fa1c41544d0 2023-11-24T06:17:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.803+0000 7fa1c8ff9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4004d00 0x7fa1c4146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.803+0000 7fa1c8ff9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4004d00 0x7fa1c4146b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59024/0 (socket says 172.21.15.5:59024) 2023-11-24T06:17:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.803+0000 7fa1c8ff9640 1 -- 172.21.15.5:0/1055138917 learned_addr learned my addr 172.21.15.5:0/1055138917 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.803+0000 7fa1c3fff640 1 --2- 172.21.15.5:0/1055138917 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c41544d0 0x7fa1c415da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.803+0000 7fa1c97fa640 1 --2- 172.21.15.5:0/1055138917 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c415df50 0x7fa1c415ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.803+0000 7fa1c8ff9640 1 -- 172.21.15.5:0/1055138917 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c41544d0 msgr2=0x7fa1c415da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.803+0000 7fa1c8ff9640 1 --2- 172.21.15.5:0/1055138917 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa1c41544d0 0x7fa1c415da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.803+0000 7fa1c8ff9640 1 -- 172.21.15.5:0/1055138917 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c415df50 msgr2=0x7fa1c415ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.803+0000 7fa1c8ff9640 1 --2- 172.21.15.5:0/1055138917 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa1c415df50 0x7fa1c415ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.803+0000 7fa1c8ff9640 1 -- 172.21.15.5:0/1055138917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c4148100 con 0x7fa1b4004d00 2023-11-24T06:17:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.803+0000 7fa1c8ff9640 1 --2- 172.21.15.5:0/1055138917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4004d00 0x7fa1c4146b50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800d570 tx=0x7fa1b800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.804+0000 7fa1c17fa640 1 -- 172.21.15.5:0/1055138917 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8005480 con 0x7fa1b4004d00 2023-11-24T06:17:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.804+0000 7fa1c17fa640 1 -- 172.21.15.5:0/1055138917 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b8003610 con 0x7fa1b4004d00 2023-11-24T06:17:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.804+0000 7fa1c17fa640 1 -- 172.21.15.5:0/1055138917 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8003980 con 0x7fa1b4004d00 2023-11-24T06:17:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.804+0000 7fa1cb25d640 1 -- 172.21.15.5:0/1055138917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c41474f0 con 0x7fa1b4004d00 2023-11-24T06:17:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.804+0000 7fa1cb25d640 1 -- 172.21.15.5:0/1055138917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c4147ab0 con 0x7fa1b4004d00 2023-11-24T06:17:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.805+0000 7fa1cb25d640 1 -- 172.21.15.5:0/1055138917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa1c40b0a30 con 0x7fa1b4004d00 2023-11-24T06:17:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.806+0000 7fa1c17fa640 1 -- 172.21.15.5:0/1055138917 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fa1b8003ae0 con 0x7fa1b4004d00 2023-11-24T06:17:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.807+0000 7fa1c17fa640 1 --2- 172.21.15.5:0/1055138917 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa1a806aed0 0x7fa1a806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.807+0000 7fa1c17fa640 1 -- 172.21.15.5:0/1055138917 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa1b8089b90 con 0x7fa1b4004d00 2023-11-24T06:17:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.807+0000 7fa1c17fa640 1 --2- 172.21.15.5:0/1055138917 >> [v2:172.21.15.6:6814/2361559103,v1:172.21.15.6:6816/2361559103] conn(0x7fa1a8070d30 0x7fa1a8073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.807+0000 7fa1c17fa640 1 -- 172.21.15.5:0/1055138917 --> [v2:172.21.15.6:6814/2361559103,v1:172.21.15.6:6816/2361559103] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa1a8073800 con 0x7fa1a8070d30 2023-11-24T06:17:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.807+0000 7fa1c17fa640 1 -- 172.21.15.5:0/1055138917 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1b808a030 con 0x7fa1b4004d00 2023-11-24T06:17:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.807+0000 7fa1c3fff640 1 --2- 172.21.15.5:0/1055138917 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa1a806aed0 0x7fa1a806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.808+0000 7fa1c97fa640 1 --2- 172.21.15.5:0/1055138917 >> [v2:172.21.15.6:6814/2361559103,v1:172.21.15.6:6816/2361559103] conn(0x7fa1a8070d30 0x7fa1a8073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.808+0000 7fa1c3fff640 1 --2- 172.21.15.5:0/1055138917 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa1a806aed0 0x7fa1a806d390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0001390 tx=0x7fa1b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.809+0000 7fa1c97fa640 1 --2- 172.21.15.5:0/1055138917 >> [v2:172.21.15.6:6814/2361559103,v1:172.21.15.6:6816/2361559103] conn(0x7fa1a8070d30 0x7fa1a8073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:57.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.811+0000 7fa1c17fa640 1 -- 172.21.15.5:0/1055138917 <== osd.6 v2:172.21.15.6:6814/2361559103 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fa1a8073800 con 0x7fa1a8070d30 2023-11-24T06:17:57.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.826+0000 7fa1cb25d640 1 -- 172.21.15.5:0/1055138917 --> [v2:172.21.15.6:6814/2361559103,v1:172.21.15.6:6816/2361559103] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa1c406e720 con 0x7fa1a8070d30 2023-11-24T06:17:57.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.827+0000 7fa1c17fa640 1 -- 172.21.15.5:0/1055138917 <== osd.6 v2:172.21.15.6:6814/2361559103 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa1bc02cb40 con 0x7fa1a8070d30 2023-11-24T06:17:57.829 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-11-24T06:17:57.829 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:17:57.829 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-11-24T06:17:57.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.827+0000 7fa1cb25d640 1 -- 172.21.15.5:0/1055138917 >> [v2:172.21.15.6:6814/2361559103,v1:172.21.15.6:6816/2361559103] conn(0x7fa1a8070d30 msgr2=0x7fa1a8073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.827+0000 7fa1cb25d640 1 --2- 172.21.15.5:0/1055138917 >> [v2:172.21.15.6:6814/2361559103,v1:172.21.15.6:6816/2361559103] conn(0x7fa1a8070d30 0x7fa1a8073150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:57.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.827+0000 7fa1cb25d640 1 -- 172.21.15.5:0/1055138917 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa1a806aed0 msgr2=0x7fa1a806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.827+0000 7fa1cb25d640 1 --2- 172.21.15.5:0/1055138917 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa1a806aed0 0x7fa1a806d390 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0001390 tx=0x7fa1b0009040 comp rx=0 tx=0).stop 2023-11-24T06:17:57.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.827+0000 7fa1cb25d640 1 -- 172.21.15.5:0/1055138917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4004d00 msgr2=0x7fa1c4146b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:57.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.827+0000 7fa1cb25d640 1 --2- 172.21.15.5:0/1055138917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1b4004d00 0x7fa1c4146b50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800d570 tx=0x7fa1b800da30 comp rx=0 tx=0).stop 2023-11-24T06:17:57.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.828+0000 7fa1c97fa640 1 -- 172.21.15.5:0/1055138917 reap_dead start 2023-11-24T06:17:57.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.828+0000 7fa1cb25d640 1 -- 172.21.15.5:0/1055138917 shutdown_connections 2023-11-24T06:17:57.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.828+0000 7fa1cb25d640 1 -- 172.21.15.5:0/1055138917 >> 172.21.15.5:0/1055138917 conn(0x7fa1c40b0ba0 msgr2=0x7fa1c40b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:57.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.828+0000 7fa1cb25d640 1 -- 172.21.15.5:0/1055138917 shutdown_connections 2023-11-24T06:17:57.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:57.828+0000 7fa1cb25d640 1 -- 172.21.15.5:0/1055138917 wait complete. 2023-11-24T06:17:57.849 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-24T06:17:58.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.065+0000 7fa5e0aea640 1 Processor -- start 2023-11-24T06:17:58.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.065+0000 7fa5e0aea640 1 -- start start 2023-11-24T06:17:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.067+0000 7fa5e0aea640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7fa5dc15ed20 0x7fa5dc141e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.067+0000 7fa5e0aea640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa5dc071820 0x7fa5dc06cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.067+0000 7fa5e0aea640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5dc06cae0 con 0x7fa5dc15f100 2023-11-24T06:17:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.067+0000 7fa5e0aea640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5dc071e40 con 0x7fa5dc071820 2023-11-24T06:17:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.067+0000 7fa5e0aea640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5dc06c1b0 con 0x7fa5dc15ed20 2023-11-24T06:17:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.068+0000 7fa5dbfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa5dc071820 0x7fa5dc06cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.068+0000 7fa5dbfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa5dc071820 0x7fa5dc06cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59040/0 (socket says 172.21.15.5:59040) 2023-11-24T06:17:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.068+0000 7fa5dbfff640 1 -- 172.21.15.5:0/3832411114 learned_addr learned my addr 172.21.15.5:0/3832411114 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.068+0000 7fa5db7fe640 1 --2- 172.21.15.5:0/3832411114 >> v2:172.21.15.6:3300/0 conn(0x7fa5dc15ed20 0x7fa5dc141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.068+0000 7fa5dbfff640 1 -- 172.21.15.5:0/3832411114 >> v2:172.21.15.6:3300/0 conn(0x7fa5dc15ed20 msgr2=0x7fa5dc141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.068+0000 7fa5dbfff640 1 --2- 172.21.15.5:0/3832411114 >> v2:172.21.15.6:3300/0 conn(0x7fa5dc15ed20 0x7fa5dc141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.068+0000 7fa5dbfff640 1 -- 172.21.15.5:0/3832411114 >> v1:172.21.15.6:6789/0 conn(0x7fa5dc15f100 legacy=0x7fa5dc071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.068+0000 7fa5dbfff640 1 -- 172.21.15.5:0/3832411114 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5dc06c490 con 0x7fa5dc071820 2023-11-24T06:17:58.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.069+0000 7fa5dbfff640 1 --2- 172.21.15.5:0/3832411114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc071820 0x7fa5dc06cf70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009e50 tx=0x7fa5d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d319c85737040bf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:58.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.070+0000 7fa5da7fc640 1 -- 172.21.15.5:0/3832411114 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d402ca80 con 0x7fa5dc071820 2023-11-24T06:17:58.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.070+0000 7fa5da7fc640 1 -- 172.21.15.5:0/3832411114 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d402cbe0 con 0x7fa5dc071820 2023-11-24T06:17:58.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.070+0000 7fa5da7fc640 1 -- 172.21.15.5:0/3832411114 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d40343f0 con 0x7fa5dc071820 2023-11-24T06:17:58.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.070+0000 7fa5e0aea640 1 -- 172.21.15.5:0/3832411114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc071820 msgr2=0x7fa5dc06cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.070+0000 7fa5e0aea640 1 --2- 172.21.15.5:0/3832411114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc071820 0x7fa5dc06cf70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4009e50 tx=0x7fa5d402bee0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.070+0000 7fa5e0aea640 1 -- 172.21.15.5:0/3832411114 shutdown_connections 2023-11-24T06:17:58.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.070+0000 7fa5e0aea640 1 --2- 172.21.15.5:0/3832411114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc071820 0x7fa5dc06cf70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.070+0000 7fa5e0aea640 1 --2- 172.21.15.5:0/3832411114 >> v2:172.21.15.6:3300/0 conn(0x7fa5dc15ed20 0x7fa5dc141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.071+0000 7fa5e0aea640 1 -- 172.21.15.5:0/3832411114 >> 172.21.15.5:0/3832411114 conn(0x7fa5dc0b0840 msgr2=0x7fa5dc0b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:58.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.071+0000 7fa5e0aea640 1 -- 172.21.15.5:0/3832411114 shutdown_connections 2023-11-24T06:17:58.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.071+0000 7fa5e0aea640 1 -- 172.21.15.5:0/3832411114 wait complete. 2023-11-24T06:17:58.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.072+0000 7fa5e0aea640 1 Processor -- start 2023-11-24T06:17:58.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.073+0000 7fa5e0aea640 1 -- start start 2023-11-24T06:17:58.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.073+0000 7fa5e0aea640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5dc071820 0x7fa5dc15d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5e0aea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc15ed20 0x7fa5dc15a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5e0aea640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc15f100 0x7fa5dc15abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5e0aea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0b29f0 con 0x7fa5dc15ed20 2023-11-24T06:17:58.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5e0aea640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0b1d70 con 0x7fa5dc15f100 2023-11-24T06:17:58.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5e0aea640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5dc0b2050 con 0x7fa5dc071820 2023-11-24T06:17:58.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5daffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc15ed20 0x7fa5dc15a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5daffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc15ed20 0x7fa5dc15a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59044/0 (socket says 172.21.15.5:59044) 2023-11-24T06:17:58.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5daffd640 1 -- 172.21.15.5:0/523376948 learned_addr learned my addr 172.21.15.5:0/523376948 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:58.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5db7fe640 1 --2- 172.21.15.5:0/523376948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5dc071820 0x7fa5dc15d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5dbfff640 1 --2- 172.21.15.5:0/523376948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc15f100 0x7fa5dc15abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5daffd640 1 -- 172.21.15.5:0/523376948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5dc071820 msgr2=0x7fa5dc15d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5daffd640 1 --2- 172.21.15.5:0/523376948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5dc071820 0x7fa5dc15d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5daffd640 1 -- 172.21.15.5:0/523376948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc15f100 msgr2=0x7fa5dc15abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5daffd640 1 --2- 172.21.15.5:0/523376948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc15f100 0x7fa5dc15abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.074+0000 7fa5daffd640 1 -- 172.21.15.5:0/523376948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d4009b30 con 0x7fa5dc15ed20 2023-11-24T06:17:58.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.075+0000 7fa5daffd640 1 --2- 172.21.15.5:0/523376948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc15ed20 0x7fa5dc15a6a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000c430 tx=0x7fa5d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:58.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.075+0000 7fa5c3fff640 1 -- 172.21.15.5:0/523376948 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0011790 con 0x7fa5dc15ed20 2023-11-24T06:17:58.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.075+0000 7fa5c3fff640 1 -- 172.21.15.5:0/523376948 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d00118f0 con 0x7fa5dc15ed20 2023-11-24T06:17:58.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.075+0000 7fa5c3fff640 1 -- 172.21.15.5:0/523376948 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d0011ce0 con 0x7fa5dc15ed20 2023-11-24T06:17:58.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.075+0000 7fa5e0aea640 1 -- 172.21.15.5:0/523376948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5dc0b2330 con 0x7fa5dc15ed20 2023-11-24T06:17:58.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.075+0000 7fa5e0aea640 1 -- 172.21.15.5:0/523376948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5dc15d890 con 0x7fa5dc15ed20 2023-11-24T06:17:58.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.077+0000 7fa5e0aea640 1 -- 172.21.15.5:0/523376948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5dc0b2b30 con 0x7fa5dc15ed20 2023-11-24T06:17:58.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.077+0000 7fa5c3fff640 1 -- 172.21.15.5:0/523376948 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fa5d001c460 con 0x7fa5dc15ed20 2023-11-24T06:17:58.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.078+0000 7fa5c3fff640 1 --2- 172.21.15.5:0/523376948 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa5c806aed0 0x7fa5c806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.078+0000 7fa5c3fff640 1 -- 172.21.15.5:0/523376948 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa5d0089370 con 0x7fa5dc15ed20 2023-11-24T06:17:58.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.079+0000 7fa5db7fe640 1 --2- 172.21.15.5:0/523376948 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa5c806aed0 0x7fa5c806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.079+0000 7fa5db7fe640 1 --2- 172.21.15.5:0/523376948 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa5c806aed0 0x7fa5c806d390 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc006fd0 tx=0x7fa5cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:58.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.083+0000 7fa5c3fff640 1 -- 172.21.15.5:0/523376948 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa5dc0b2b30 con 0x7fa5dc15ed20 2023-11-24T06:17:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.280+0000 7fa5e0aea640 1 -- 172.21.15.5:0/523376948 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5dc0b0c20 con 0x7fa5c806aed0 2023-11-24T06:17:58.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.281+0000 7fa5c3fff640 1 -- 172.21.15.5:0/523376948 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa5dc0b0c20 con 0x7fa5c806aed0 2023-11-24T06:17:58.283 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 6 to deep-scrub 2023-11-24T06:17:58.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.284+0000 7fa5e0aea640 1 -- 172.21.15.5:0/523376948 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa5c806aed0 msgr2=0x7fa5c806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.284+0000 7fa5e0aea640 1 --2- 172.21.15.5:0/523376948 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa5c806aed0 0x7fa5c806d390 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc006fd0 tx=0x7fa5cc008040 comp rx=0 tx=0).stop 2023-11-24T06:17:58.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.284+0000 7fa5e0aea640 1 -- 172.21.15.5:0/523376948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc15ed20 msgr2=0x7fa5dc15a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.284+0000 7fa5e0aea640 1 --2- 172.21.15.5:0/523376948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc15ed20 0x7fa5dc15a6a0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa5d000c430 tx=0x7fa5d000c8f0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.284+0000 7fa5e0aea640 1 -- 172.21.15.5:0/523376948 shutdown_connections 2023-11-24T06:17:58.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.284+0000 7fa5e0aea640 1 --2- 172.21.15.5:0/523376948 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fa5c806aed0 0x7fa5c806d390 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.284+0000 7fa5e0aea640 1 --2- 172.21.15.5:0/523376948 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fa5dc15f100 0x7fa5dc15abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.284+0000 7fa5e0aea640 1 --2- 172.21.15.5:0/523376948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5dc15ed20 0x7fa5dc15a6a0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.284+0000 7fa5e0aea640 1 --2- 172.21.15.5:0/523376948 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fa5dc071820 0x7fa5dc15d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.284+0000 7fa5e0aea640 1 -- 172.21.15.5:0/523376948 >> 172.21.15.5:0/523376948 conn(0x7fa5dc0b0840 msgr2=0x7fa5dc0affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.284+0000 7fa5e0aea640 1 -- 172.21.15.5:0/523376948 shutdown_connections 2023-11-24T06:17:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.284+0000 7fa5e0aea640 1 -- 172.21.15.5:0/523376948 wait complete. 2023-11-24T06:17:58.306 INFO:tasks.ceph:Scrubbing osd.7 2023-11-24T06:17:58.307 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-24T06:17:58.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.532+0000 7fe438dff640 1 Processor -- start 2023-11-24T06:17:58.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.532+0000 7fe438dff640 1 -- start start 2023-11-24T06:17:58.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.534+0000 7fe438dff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe43415f270 0x7fe4340cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.534+0000 7fe438dff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe43406d9e0 0x7fe4340cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.534+0000 7fe438dff640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe43415f930 con 0x7fe434144770 2023-11-24T06:17:58.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.534+0000 7fe438dff640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4340b43b0 con 0x7fe43406d9e0 2023-11-24T06:17:58.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.534+0000 7fe438dff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4340b3b10 con 0x7fe43415f270 2023-11-24T06:17:58.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.534+0000 7fe433fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe43406d9e0 0x7fe4340cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.534+0000 7fe4337fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe434144770 0x7fe43415eb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50710/0 (socket says 172.21.15.5:50710) 2023-11-24T06:17:58.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.534+0000 7fe433fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe43406d9e0 0x7fe4340cd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59050/0 (socket says 172.21.15.5:59050) 2023-11-24T06:17:58.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.534+0000 7fe4337fe640 1 -- 172.21.15.5:0/1486694286 learned_addr learned my addr 172.21.15.5:0/1486694286 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:58.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.534+0000 7fe432ffd640 1 --2- 172.21.15.5:0/1486694286 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe43415f270 0x7fe4340cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.535+0000 7fe433fff640 1 -- 172.21.15.5:0/1486694286 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe43415f270 msgr2=0x7fe4340cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:58.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.535+0000 7fe433fff640 1 --2- 172.21.15.5:0/1486694286 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe43415f270 0x7fe4340cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.535+0000 7fe433fff640 1 -- 172.21.15.5:0/1486694286 >> v1:172.21.15.5:6789/0 conn(0x7fe434144770 legacy=0x7fe43415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.535+0000 7fe433fff640 1 -- 172.21.15.5:0/1486694286 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4340b3df0 con 0x7fe43406d9e0 2023-11-24T06:17:58.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.535+0000 7fe4327fc640 1 -- 172.21.15.5:0/1486694286 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693983113 0 0) 0x7fe43415f930 con 0x7fe434144770 2023-11-24T06:17:58.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.536+0000 7fe433fff640 1 --2- 172.21.15.5:0/1486694286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe43406d9e0 0x7fe4340cd9e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0097a0 tx=0x7fe42c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1207b5752aec25 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:58.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.536+0000 7fe4327fc640 1 -- 172.21.15.5:0/1486694286 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c02d040 con 0x7fe43406d9e0 2023-11-24T06:17:58.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.536+0000 7fe4327fc640 1 -- 172.21.15.5:0/1486694286 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42c031810 con 0x7fe43406d9e0 2023-11-24T06:17:58.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.536+0000 7fe4327fc640 1 -- 172.21.15.5:0/1486694286 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c031ba0 con 0x7fe43406d9e0 2023-11-24T06:17:58.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.537+0000 7fe438dff640 1 -- 172.21.15.5:0/1486694286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe43406d9e0 msgr2=0x7fe4340cd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.537+0000 7fe438dff640 1 --2- 172.21.15.5:0/1486694286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe43406d9e0 0x7fe4340cd9e0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0097a0 tx=0x7fe42c02b8b0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.537+0000 7fe438dff640 1 -- 172.21.15.5:0/1486694286 shutdown_connections 2023-11-24T06:17:58.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.537+0000 7fe438dff640 1 --2- 172.21.15.5:0/1486694286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe43406d9e0 0x7fe4340cd9e0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.537+0000 7fe438dff640 1 --2- 172.21.15.5:0/1486694286 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe43415f270 0x7fe4340cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.537+0000 7fe438dff640 1 -- 172.21.15.5:0/1486694286 >> 172.21.15.5:0/1486694286 conn(0x7fe4340cec10 msgr2=0x7fe4340cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:58.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.537+0000 7fe438dff640 1 -- 172.21.15.5:0/1486694286 shutdown_connections 2023-11-24T06:17:58.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.537+0000 7fe438dff640 1 -- 172.21.15.5:0/1486694286 wait complete. 2023-11-24T06:17:58.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.539+0000 7fe438dff640 1 Processor -- start 2023-11-24T06:17:58.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.539+0000 7fe438dff640 1 -- start start 2023-11-24T06:17:58.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.540+0000 7fe438dff640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe43406d9e0 0x7fe4341f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.540+0000 7fe438dff640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe434144770 0x7fe43415dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.540+0000 7fe438dff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe43415f270 0x7fe43413a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.540+0000 7fe438dff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4341f3370 con 0x7fe43415f270 2023-11-24T06:17:58.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.540+0000 7fe438dff640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43413fa40 con 0x7fe43406d9e0 2023-11-24T06:17:58.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.540+0000 7fe438dff640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43413fd20 con 0x7fe434144770 2023-11-24T06:17:58.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.541+0000 7fe433fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe43415f270 0x7fe43413a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.541+0000 7fe433fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe43415f270 0x7fe43413a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59058/0 (socket says 172.21.15.5:59058) 2023-11-24T06:17:58.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.541+0000 7fe433fff640 1 -- 172.21.15.5:0/1931246536 learned_addr learned my addr 172.21.15.5:0/1931246536 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:58.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.541+0000 7fe432ffd640 1 --2- 172.21.15.5:0/1931246536 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe434144770 0x7fe43415dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.541+0000 7fe4337fe640 1 --2- 172.21.15.5:0/1931246536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe43406d9e0 0x7fe4341f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.541+0000 7fe433fff640 1 -- 172.21.15.5:0/1931246536 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe434144770 msgr2=0x7fe43415dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.541+0000 7fe433fff640 1 --2- 172.21.15.5:0/1931246536 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7fe434144770 0x7fe43415dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.541+0000 7fe433fff640 1 -- 172.21.15.5:0/1931246536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe43406d9e0 msgr2=0x7fe4341f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.541+0000 7fe433fff640 1 --2- 172.21.15.5:0/1931246536 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7fe43406d9e0 0x7fe4341f2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.541+0000 7fe433fff640 1 -- 172.21.15.5:0/1931246536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c009400 con 0x7fe43415f270 2023-11-24T06:17:58.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.542+0000 7fe433fff640 1 --2- 172.21.15.5:0/1931246536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe43415f270 0x7fe43413a5f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0097a0 tx=0x7fe42c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:58.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.542+0000 7fe42bfff640 1 -- 172.21.15.5:0/1931246536 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c006e10 con 0x7fe43415f270 2023-11-24T06:17:58.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.542+0000 7fe42bfff640 1 -- 172.21.15.5:0/1931246536 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42c03bb60 con 0x7fe43415f270 2023-11-24T06:17:58.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.542+0000 7fe438dff640 1 -- 172.21.15.5:0/1931246536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe434140000 con 0x7fe43415f270 2023-11-24T06:17:58.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.542+0000 7fe42bfff640 1 -- 172.21.15.5:0/1931246536 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c004030 con 0x7fe43415f270 2023-11-24T06:17:58.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.542+0000 7fe438dff640 1 -- 172.21.15.5:0/1931246536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4341405c0 con 0x7fe43415f270 2023-11-24T06:17:58.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.543+0000 7fe438dff640 1 -- 172.21.15.5:0/1931246536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe434144b50 con 0x7fe43415f270 2023-11-24T06:17:58.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.544+0000 7fe42bfff640 1 -- 172.21.15.5:0/1931246536 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe42c043070 con 0x7fe43415f270 2023-11-24T06:17:58.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.545+0000 7fe42bfff640 1 --2- 172.21.15.5:0/1931246536 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe41406aed0 0x7fe41406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.545+0000 7fe42bfff640 1 -- 172.21.15.5:0/1931246536 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe42c0ab4d0 con 0x7fe43415f270 2023-11-24T06:17:58.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.545+0000 7fe42bfff640 1 --2- 172.21.15.5:0/1931246536 >> [v2:172.21.15.6:6808/4073132861,v1:172.21.15.6:6809/4073132861] conn(0x7fe414070d30 0x7fe414073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.545+0000 7fe42bfff640 1 -- 172.21.15.5:0/1931246536 --> [v2:172.21.15.6:6808/4073132861,v1:172.21.15.6:6809/4073132861] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe414073800 con 0x7fe414070d30 2023-11-24T06:17:58.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.545+0000 7fe42bfff640 1 -- 172.21.15.5:0/1931246536 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe42c0adab0 con 0x7fe43415f270 2023-11-24T06:17:58.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.546+0000 7fe4337fe640 1 --2- 172.21.15.5:0/1931246536 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe41406aed0 0x7fe41406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.546+0000 7fe432ffd640 1 --2- 172.21.15.5:0/1931246536 >> [v2:172.21.15.6:6808/4073132861,v1:172.21.15.6:6809/4073132861] conn(0x7fe414070d30 0x7fe414073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.546+0000 7fe4337fe640 1 --2- 172.21.15.5:0/1931246536 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe41406aed0 0x7fe41406d390 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4240012e0 tx=0x7fe424007ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:58.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.547+0000 7fe432ffd640 1 --2- 172.21.15.5:0/1931246536 >> [v2:172.21.15.6:6808/4073132861,v1:172.21.15.6:6809/4073132861] conn(0x7fe414070d30 0x7fe414073150 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:58.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.549+0000 7fe42bfff640 1 -- 172.21.15.5:0/1931246536 <== osd.7 v2:172.21.15.6:6808/4073132861 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe414073800 con 0x7fe414070d30 2023-11-24T06:17:58.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.564+0000 7fe438dff640 1 -- 172.21.15.5:0/1931246536 --> [v2:172.21.15.6:6808/4073132861,v1:172.21.15.6:6809/4073132861] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe43406c4b0 con 0x7fe414070d30 2023-11-24T06:17:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.565+0000 7fe42bfff640 1 -- 172.21.15.5:0/1931246536 <== osd.7 v2:172.21.15.6:6808/4073132861 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe41c00b520 con 0x7fe414070d30 2023-11-24T06:17:58.567 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-11-24T06:17:58.567 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T06:17:58.567 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-11-24T06:17:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.565+0000 7fe438dff640 1 -- 172.21.15.5:0/1931246536 >> [v2:172.21.15.6:6808/4073132861,v1:172.21.15.6:6809/4073132861] conn(0x7fe414070d30 msgr2=0x7fe414073150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.565+0000 7fe438dff640 1 --2- 172.21.15.5:0/1931246536 >> [v2:172.21.15.6:6808/4073132861,v1:172.21.15.6:6809/4073132861] conn(0x7fe414070d30 0x7fe414073150 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.565+0000 7fe438dff640 1 -- 172.21.15.5:0/1931246536 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe41406aed0 msgr2=0x7fe41406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.565+0000 7fe438dff640 1 --2- 172.21.15.5:0/1931246536 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7fe41406aed0 0x7fe41406d390 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4240012e0 tx=0x7fe424007ed0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.565+0000 7fe438dff640 1 -- 172.21.15.5:0/1931246536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe43415f270 msgr2=0x7fe43413a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.565+0000 7fe438dff640 1 --2- 172.21.15.5:0/1931246536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe43415f270 0x7fe43413a5f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe42c0097a0 tx=0x7fe42c0046c0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.565+0000 7fe433fff640 1 -- 172.21.15.5:0/1931246536 reap_dead start 2023-11-24T06:17:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.565+0000 7fe438dff640 1 -- 172.21.15.5:0/1931246536 shutdown_connections 2023-11-24T06:17:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.565+0000 7fe438dff640 1 -- 172.21.15.5:0/1931246536 >> 172.21.15.5:0/1931246536 conn(0x7fe4340cec10 msgr2=0x7fe43415e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:58.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.565+0000 7fe438dff640 1 -- 172.21.15.5:0/1931246536 shutdown_connections 2023-11-24T06:17:58.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.565+0000 7fe438dff640 1 -- 172.21.15.5:0/1931246536 wait complete. 2023-11-24T06:17:58.584 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-24T06:17:58.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.805+0000 7f4ece252640 1 Processor -- start 2023-11-24T06:17:58.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.806+0000 7f4ece252640 1 -- start start 2023-11-24T06:17:58.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.807+0000 7f4ece252640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ec815d430 0x7f4ec80d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.807+0000 7f4ece252640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4ec80d1690 0x7f4ec80d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.807+0000 7f4ece252640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec80cff00 con 0x7f4ec80cfaa0 2023-11-24T06:17:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.807+0000 7f4ece252640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec80d09d0 con 0x7f4ec80d1690 2023-11-24T06:17:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.807+0000 7f4ece252640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec80d1030 con 0x7f4ec815d430 2023-11-24T06:17:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.807+0000 7f4ecda51640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4ec80d1690 0x7f4ec80d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.807+0000 7f4ecca4f640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4ec80cfaa0 0x7f4ec81615e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50712/0 (socket says 172.21.15.5:50712) 2023-11-24T06:17:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.808+0000 7f4ecca4f640 1 -- 172.21.15.5:0/3208135979 learned_addr learned my addr 172.21.15.5:0/3208135979 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.808+0000 7f4ecd250640 1 --2- 172.21.15.5:0/3208135979 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ec815d430 0x7f4ec80d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.808+0000 7f4ecda51640 1 -- 172.21.15.5:0/3208135979 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ec815d430 msgr2=0x7f4ec80d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.808+0000 7f4ecda51640 1 --2- 172.21.15.5:0/3208135979 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ec815d430 0x7f4ec80d3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.808+0000 7f4ecda51640 1 -- 172.21.15.5:0/3208135979 >> v1:172.21.15.5:6789/0 conn(0x7f4ec80cfaa0 legacy=0x7f4ec81615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.808+0000 7f4ecda51640 1 -- 172.21.15.5:0/3208135979 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec80d1fe0 con 0x7f4ec80d1690 2023-11-24T06:17:58.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.808+0000 7f4ebffff640 1 -- 172.21.15.5:0/3208135979 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758064608 0 0) 0x7f4ec80cff00 con 0x7f4ec80cfaa0 2023-11-24T06:17:58.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.809+0000 7f4ecda51640 1 --2- 172.21.15.5:0/3208135979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec80d1690 0x7f4ec80d1aa0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009d90 tx=0x7f4ec002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15e94550b411249 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:58.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.810+0000 7f4ebffff640 1 -- 172.21.15.5:0/3208135979 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec002cbc0 con 0x7f4ec80d1690 2023-11-24T06:17:58.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.810+0000 7f4ebffff640 1 -- 172.21.15.5:0/3208135979 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec002cd20 con 0x7f4ec80d1690 2023-11-24T06:17:58.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.810+0000 7f4ebffff640 1 -- 172.21.15.5:0/3208135979 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0034660 con 0x7f4ec80d1690 2023-11-24T06:17:58.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.810+0000 7f4ece252640 1 -- 172.21.15.5:0/3208135979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec80d1690 msgr2=0x7f4ec80d1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.810+0000 7f4ece252640 1 --2- 172.21.15.5:0/3208135979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec80d1690 0x7f4ec80d1aa0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0009d90 tx=0x7f4ec002be10 comp rx=0 tx=0).stop 2023-11-24T06:17:58.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.810+0000 7f4ece252640 1 -- 172.21.15.5:0/3208135979 shutdown_connections 2023-11-24T06:17:58.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.810+0000 7f4ece252640 1 --2- 172.21.15.5:0/3208135979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec80d1690 0x7f4ec80d1aa0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.810+0000 7f4ece252640 1 --2- 172.21.15.5:0/3208135979 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ec815d430 0x7f4ec80d3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.810+0000 7f4ece252640 1 -- 172.21.15.5:0/3208135979 >> 172.21.15.5:0/3208135979 conn(0x7f4ec808cb00 msgr2=0x7f4ec809c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:58.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.810+0000 7f4ece252640 1 -- 172.21.15.5:0/3208135979 shutdown_connections 2023-11-24T06:17:58.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.810+0000 7f4ece252640 1 -- 172.21.15.5:0/3208135979 wait complete. 2023-11-24T06:17:58.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.812+0000 7f4ece252640 1 Processor -- start 2023-11-24T06:17:58.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.812+0000 7f4ece252640 1 -- start start 2023-11-24T06:17:58.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.813+0000 7f4ece252640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ec80cfaa0 0x7f4ec81510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.813+0000 7f4ece252640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec80d1690 0x7f4ec80b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.813+0000 7f4ece252640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ec815d430 0x7f4ec80b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.813+0000 7f4ece252640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec8151620 con 0x7f4ec80d1690 2023-11-24T06:17:58.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.813+0000 7f4ece252640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec815e790 con 0x7f4ec80cfaa0 2023-11-24T06:17:58.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.813+0000 7f4ece252640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec815dbc0 con 0x7f4ec815d430 2023-11-24T06:17:58.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.814+0000 7f4ecca4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec80d1690 0x7f4ec80b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.814+0000 7f4ecca4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec80d1690 0x7f4ec80b6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59084/0 (socket says 172.21.15.5:59084) 2023-11-24T06:17:58.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.814+0000 7f4ecca4f640 1 -- 172.21.15.5:0/3273068785 learned_addr learned my addr 172.21.15.5:0/3273068785 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:58.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.814+0000 7f4ecda51640 1 --2- 172.21.15.5:0/3273068785 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ec815d430 0x7f4ec80b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.814+0000 7f4ecd250640 1 --2- 172.21.15.5:0/3273068785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ec80cfaa0 0x7f4ec81510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.814+0000 7f4ecca4f640 1 -- 172.21.15.5:0/3273068785 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ec815d430 msgr2=0x7f4ec80b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.814+0000 7f4ecca4f640 1 --2- 172.21.15.5:0/3273068785 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ec815d430 0x7f4ec80b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.814+0000 7f4ecca4f640 1 -- 172.21.15.5:0/3273068785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ec80cfaa0 msgr2=0x7f4ec81510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:58.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.814+0000 7f4ecca4f640 1 --2- 172.21.15.5:0/3273068785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ec80cfaa0 0x7f4ec81510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:58.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.814+0000 7f4ecca4f640 1 -- 172.21.15.5:0/3273068785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec0009a70 con 0x7f4ec80d1690 2023-11-24T06:17:58.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.814+0000 7f4ecca4f640 1 --2- 172.21.15.5:0/3273068785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec80d1690 0x7f4ec80b6aa0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800c4d0 tx=0x7f4eb800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:58.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.815+0000 7f4ebdffb640 1 -- 172.21.15.5:0/3273068785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb80074e0 con 0x7f4ec80d1690 2023-11-24T06:17:58.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.815+0000 7f4ebdffb640 1 -- 172.21.15.5:0/3273068785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb8007660 con 0x7f4ec80d1690 2023-11-24T06:17:58.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.815+0000 7f4ebdffb640 1 -- 172.21.15.5:0/3273068785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8004a20 con 0x7f4ec80d1690 2023-11-24T06:17:58.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.815+0000 7f4ece252640 1 -- 172.21.15.5:0/3273068785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec815de60 con 0x7f4ec80d1690 2023-11-24T06:17:58.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.815+0000 7f4ece252640 1 -- 172.21.15.5:0/3273068785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec815e420 con 0x7f4ec80d1690 2023-11-24T06:17:58.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.816+0000 7f4ece252640 1 -- 172.21.15.5:0/3273068785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec8067960 con 0x7f4ec80d1690 2023-11-24T06:17:58.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.817+0000 7f4ebdffb640 1 -- 172.21.15.5:0/3273068785 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f4eb8004b80 con 0x7f4ec80d1690 2023-11-24T06:17:58.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.818+0000 7f4ebdffb640 1 --2- 172.21.15.5:0/3273068785 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4eac06aed0 0x7f4eac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:58.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.818+0000 7f4ebdffb640 1 -- 172.21.15.5:0/3273068785 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4eb80884e0 con 0x7f4ec80d1690 2023-11-24T06:17:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.818+0000 7f4ecd250640 1 --2- 172.21.15.5:0/3273068785 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4eac06aed0 0x7f4eac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:58.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.819+0000 7f4ecd250640 1 --2- 172.21.15.5:0/3273068785 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4eac06aed0 0x7f4eac06d390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4006fd0 tx=0x7f4ec4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:58.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:58.822+0000 7f4ebdffb640 1 -- 172.21.15.5:0/3273068785 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4eb80538a0 con 0x7f4ec80d1690 2023-11-24T06:17:59.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.017+0000 7f4ece252640 1 -- 172.21.15.5:0/3273068785 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ec80d0410 con 0x7f4eac06aed0 2023-11-24T06:17:59.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.017+0000 7f4ebdffb640 1 -- 172.21.15.5:0/3273068785 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4ec400b5e0 con 0x7f4eac06aed0 2023-11-24T06:17:59.019 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 7 to deep-scrub 2023-11-24T06:17:59.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.020+0000 7f4ece252640 1 -- 172.21.15.5:0/3273068785 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4eac06aed0 msgr2=0x7f4eac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.020+0000 7f4ece252640 1 --2- 172.21.15.5:0/3273068785 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4eac06aed0 0x7f4eac06d390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4006fd0 tx=0x7f4ec4008040 comp rx=0 tx=0).stop 2023-11-24T06:17:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.020+0000 7f4ece252640 1 -- 172.21.15.5:0/3273068785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec80d1690 msgr2=0x7f4ec80b6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.020+0000 7f4ece252640 1 --2- 172.21.15.5:0/3273068785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec80d1690 0x7f4ec80b6aa0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800c4d0 tx=0x7f4eb800c990 comp rx=0 tx=0).stop 2023-11-24T06:17:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.021+0000 7f4ece252640 1 -- 172.21.15.5:0/3273068785 shutdown_connections 2023-11-24T06:17:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.021+0000 7f4ece252640 1 --2- 172.21.15.5:0/3273068785 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4eac06aed0 0x7f4eac06d390 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.021+0000 7f4ece252640 1 --2- 172.21.15.5:0/3273068785 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4ec815d430 0x7f4ec80b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.021+0000 7f4ece252640 1 --2- 172.21.15.5:0/3273068785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ec80d1690 0x7f4ec80b6aa0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.021+0000 7f4ece252640 1 --2- 172.21.15.5:0/3273068785 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4ec80cfaa0 0x7f4ec81510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.021+0000 7f4ece252640 1 -- 172.21.15.5:0/3273068785 >> 172.21.15.5:0/3273068785 conn(0x7f4ec808cb00 msgr2=0x7f4ec8155940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.021+0000 7f4ece252640 1 -- 172.21.15.5:0/3273068785 shutdown_connections 2023-11-24T06:17:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.021+0000 7f4ece252640 1 -- 172.21.15.5:0/3273068785 wait complete. 2023-11-24T06:17:59.041 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:17:59.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.259+0000 7f8c52bbc640 1 Processor -- start 2023-11-24T06:17:59.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.259+0000 7f8c52bbc640 1 -- start start 2023-11-24T06:17:59.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.260+0000 7f8c52bbc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c4c15d960 0x7f8c4c0d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:59.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.260+0000 7f8c52bbc640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8c4c0d1be0 0x7f8c4c0b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:59.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.261+0000 7f8c52bbc640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4c0b21b0 con 0x7f8c4c15d580 2023-11-24T06:17:59.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.261+0000 7f8c52bbc640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4c0d2070 con 0x7f8c4c0d1be0 2023-11-24T06:17:59.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.261+0000 7f8c52bbc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0afde0 con 0x7f8c4c15d960 2023-11-24T06:17:59.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.261+0000 7f8c51bba640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8c4c15d580 0x7f8c4c0d37c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:50716/0 (socket says 172.21.15.5:50716) 2023-11-24T06:17:59.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.261+0000 7f8c51bba640 1 -- 172.21.15.5:0/4074903160 learned_addr learned my addr 172.21.15.5:0/4074903160 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:17:59.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.261+0000 7f8c513b9640 1 --2- 172.21.15.5:0/4074903160 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c4c15d960 0x7f8c4c0d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:59.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.261+0000 7f8c523bb640 1 --2- 172.21.15.5:0/4074903160 >> v2:172.21.15.6:3300/0 conn(0x7f8c4c0d1be0 0x7f8c4c0b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:59.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.261+0000 7f8c50bb8640 1 -- 172.21.15.5:0/4074903160 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621064599 0 0) 0x7f8c4c0b21b0 con 0x7f8c4c15d580 2023-11-24T06:17:59.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.261+0000 7f8c50bb8640 1 -- 172.21.15.5:0/4074903160 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c38003520 con 0x7f8c4c15d580 2023-11-24T06:17:59.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c50bb8640 1 -- 172.21.15.5:0/4074903160 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2165671792 0 0) 0x7f8c38003520 con 0x7f8c4c15d580 2023-11-24T06:17:59.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c50bb8640 1 -- 172.21.15.5:0/4074903160 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c4c15d960 msgr2=0x7f8c4c0d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:59.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c50bb8640 1 --2- 172.21.15.5:0/4074903160 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c4c15d960 0x7f8c4c0d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c50bb8640 1 -- 172.21.15.5:0/4074903160 >> v2:172.21.15.6:3300/0 conn(0x7f8c4c0d1be0 msgr2=0x7f8c4c0b0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:59.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c50bb8640 1 --2- 172.21.15.5:0/4074903160 >> v2:172.21.15.6:3300/0 conn(0x7f8c4c0d1be0 0x7f8c4c0b0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c50bb8640 1 -- 172.21.15.5:0/4074903160 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c0b00c0 con 0x7f8c4c15d580 2023-11-24T06:17:59.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c50bb8640 1 -- 172.21.15.5:0/4074903160 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f8c48001670 con 0x7f8c4c15d580 2023-11-24T06:17:59.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c523bb640 1 --2- 172.21.15.5:0/4074903160 >> v2:172.21.15.6:3300/0 conn(0x7f8c4c0d1be0 0x7f8c4c0b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T06:17:59.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c50bb8640 1 -- 172.21.15.5:0/4074903160 >> v1:172.21.15.5:6789/0 conn(0x7f8c4c15d580 legacy=0x7f8c4c0d37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:59.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c50bb8640 1 --2- 172.21.15.5:0/4074903160 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c4c15d960 0x7f8c380039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:59.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c50bb8640 1 --2- 172.21.15.5:0/4074903160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c38003f60 0x7f8c380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:59.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c50bb8640 1 --2- 172.21.15.5:0/4074903160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c38004d00 0x7f8c38005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:59.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c50bb8640 1 -- 172.21.15.5:0/4074903160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0d2070 con 0x7f8c38004d00 2023-11-24T06:17:59.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c50bb8640 1 -- 172.21.15.5:0/4074903160 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c0afde0 con 0x7f8c38003f60 2023-11-24T06:17:59.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c50bb8640 1 -- 172.21.15.5:0/4074903160 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3800ba60 con 0x7f8c4c15d960 2023-11-24T06:17:59.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.262+0000 7f8c523bb640 1 --2- 172.21.15.5:0/4074903160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c38004d00 0x7f8c38005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:59.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.263+0000 7f8c51bba640 1 --2- 172.21.15.5:0/4074903160 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c4c15d960 0x7f8c380039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:59.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.263+0000 7f8c513b9640 1 --2- 172.21.15.5:0/4074903160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c38003f60 0x7f8c380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:59.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.263+0000 7f8c523bb640 1 -- 172.21.15.5:0/4074903160 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c4c15d960 msgr2=0x7f8c380039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:59.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.263+0000 7f8c523bb640 1 --2- 172.21.15.5:0/4074903160 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c4c15d960 0x7f8c380039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.263+0000 7f8c523bb640 1 -- 172.21.15.5:0/4074903160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c38003f60 msgr2=0x7f8c380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:17:59.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.263+0000 7f8c523bb640 1 --2- 172.21.15.5:0/4074903160 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c38003f60 0x7f8c380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.263+0000 7f8c523bb640 1 -- 172.21.15.5:0/4074903160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c38003520 con 0x7f8c38004d00 2023-11-24T06:17:59.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.264+0000 7f8c523bb640 1 --2- 172.21.15.5:0/4074903160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c38004d00 0x7f8c38005180 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8c440073e0 tx=0x7f8c380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c04a97a7c15187d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:59.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.264+0000 7f8c50bb8640 1 -- 172.21.15.5:0/4074903160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4402cdc0 con 0x7f8c38004d00 2023-11-24T06:17:59.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.265+0000 7f8c50bb8640 1 -- 172.21.15.5:0/4074903160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c44034440 con 0x7f8c38004d00 2023-11-24T06:17:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.265+0000 7f8c50bb8640 1 -- 172.21.15.5:0/4074903160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c440347b0 con 0x7f8c38004d00 2023-11-24T06:17:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.265+0000 7f8c52bbc640 1 -- 172.21.15.5:0/4074903160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c38004d00 msgr2=0x7f8c38005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.265+0000 7f8c52bbc640 1 --2- 172.21.15.5:0/4074903160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c38004d00 0x7f8c38005180 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8c440073e0 tx=0x7f8c380057f0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.265+0000 7f8c523bb640 1 -- 172.21.15.5:0/4074903160 reap_dead start 2023-11-24T06:17:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.265+0000 7f8c52bbc640 1 -- 172.21.15.5:0/4074903160 shutdown_connections 2023-11-24T06:17:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.265+0000 7f8c52bbc640 1 -- 172.21.15.5:0/4074903160 >> 172.21.15.5:0/4074903160 conn(0x7f8c4c09c3e0 msgr2=0x7f8c4c0d30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.265+0000 7f8c52bbc640 1 -- 172.21.15.5:0/4074903160 shutdown_connections 2023-11-24T06:17:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.265+0000 7f8c52bbc640 1 -- 172.21.15.5:0/4074903160 wait complete. 2023-11-24T06:17:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.267+0000 7f8c52bbc640 1 Processor -- start 2023-11-24T06:17:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.267+0000 7f8c52bbc640 1 -- start start 2023-11-24T06:17:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.268+0000 7f8c52bbc640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c38004d00 0x7f8c4c1524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:59.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.268+0000 7f8c52bbc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c4c1603a0 0x7f8c4c0b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:59.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.268+0000 7f8c52bbc640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c4c160820 0x7f8c4c0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:59.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.268+0000 7f8c52bbc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c15e750 con 0x7f8c4c1603a0 2023-11-24T06:17:59.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.268+0000 7f8c52bbc640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c15ea30 con 0x7f8c4c160820 2023-11-24T06:17:59.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.268+0000 7f8c52bbc640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c15ed10 con 0x7f8c38004d00 2023-11-24T06:17:59.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.269+0000 7f8c513b9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c4c1603a0 0x7f8c4c0b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.269+0000 7f8c513b9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c4c1603a0 0x7f8c4c0b5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59096/0 (socket says 172.21.15.5:59096) 2023-11-24T06:17:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.269+0000 7f8c513b9640 1 -- 172.21.15.5:0/124758423 learned_addr learned my addr 172.21.15.5:0/124758423 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:17:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.269+0000 7f8c51bba640 1 --2- 172.21.15.5:0/124758423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c38004d00 0x7f8c4c1524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.269+0000 7f8c513b9640 1 -- 172.21.15.5:0/124758423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c38004d00 msgr2=0x7f8c4c1524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.269+0000 7f8c513b9640 1 --2- 172.21.15.5:0/124758423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c38004d00 0x7f8c4c1524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.269+0000 7f8c513b9640 1 -- 172.21.15.5:0/124758423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c4c160820 msgr2=0x7f8c4c0b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:59.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.269+0000 7f8c513b9640 1 --2- 172.21.15.5:0/124758423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c4c160820 0x7f8c4c0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.269+0000 7f8c513b9640 1 -- 172.21.15.5:0/124758423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c15eff0 con 0x7f8c4c1603a0 2023-11-24T06:17:59.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.269+0000 7f8c513b9640 1 --2- 172.21.15.5:0/124758423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c4c1603a0 0x7f8c4c0b5c30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c011540 tx=0x7f8c3c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:59.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.270+0000 7f8c427fc640 1 -- 172.21.15.5:0/124758423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c015930 con 0x7f8c4c1603a0 2023-11-24T06:17:59.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.270+0000 7f8c427fc640 1 -- 172.21.15.5:0/124758423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c015a90 con 0x7f8c4c1603a0 2023-11-24T06:17:59.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.270+0000 7f8c427fc640 1 -- 172.21.15.5:0/124758423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c015e60 con 0x7f8c4c1603a0 2023-11-24T06:17:59.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.270+0000 7f8c52bbc640 1 -- 172.21.15.5:0/124758423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4c15f2d0 con 0x7f8c4c1603a0 2023-11-24T06:17:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.270+0000 7f8c52bbc640 1 -- 172.21.15.5:0/124758423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4c0b8650 con 0x7f8c4c1603a0 2023-11-24T06:17:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.271+0000 7f8c52bbc640 1 -- 172.21.15.5:0/124758423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4c04e840 con 0x7f8c4c1603a0 2023-11-24T06:17:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.272+0000 7f8c427fc640 1 -- 172.21.15.5:0/124758423 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8c3c01d400 con 0x7f8c4c1603a0 2023-11-24T06:17:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.273+0000 7f8c427fc640 1 --2- 172.21.15.5:0/124758423 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c3006aed0 0x7f8c3006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:17:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.273+0000 7f8c427fc640 1 -- 172.21.15.5:0/124758423 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8c3c089bd0 con 0x7f8c4c1603a0 2023-11-24T06:17:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.273+0000 7f8c51bba640 1 --2- 172.21.15.5:0/124758423 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c3006aed0 0x7f8c3006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:17:59.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.274+0000 7f8c51bba640 1 --2- 172.21.15.5:0/124758423 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c3006aed0 0x7f8c3006d390 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8c48000fb0 tx=0x7f8c48009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:17:59.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.277+0000 7f8c427fc640 1 -- 172.21.15.5:0/124758423 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c3c054e90 con 0x7f8c4c1603a0 2023-11-24T06:17:59.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.470+0000 7f8c52bbc640 1 -- 172.21.15.5:0/124758423 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8c4c15ee80 con 0x7f8c3006aed0 2023-11-24T06:17:59.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.475+0000 7f8c427fc640 1 -- 172.21.15.5:0/124758423 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48557 (secure 0 0 0) 0x7f8c4c15ee80 con 0x7f8c3006aed0 2023-11-24T06:17:59.478 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:17:59.478 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-11-24T06:17:59.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.478+0000 7f8c52bbc640 1 -- 172.21.15.5:0/124758423 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c3006aed0 msgr2=0x7f8c3006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:59.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.478+0000 7f8c52bbc640 1 --2- 172.21.15.5:0/124758423 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c3006aed0 0x7f8c3006d390 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8c48000fb0 tx=0x7f8c48009040 comp rx=0 tx=0).stop 2023-11-24T06:17:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.478+0000 7f8c52bbc640 1 -- 172.21.15.5:0/124758423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c4c1603a0 msgr2=0x7f8c4c0b5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:17:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.478+0000 7f8c52bbc640 1 --2- 172.21.15.5:0/124758423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c4c1603a0 0x7f8c4c0b5c30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c011540 tx=0x7f8c3c011a00 comp rx=0 tx=0).stop 2023-11-24T06:17:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.478+0000 7f8c52bbc640 1 -- 172.21.15.5:0/124758423 shutdown_connections 2023-11-24T06:17:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.478+0000 7f8c52bbc640 1 --2- 172.21.15.5:0/124758423 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c3006aed0 0x7f8c3006d390 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.478+0000 7f8c52bbc640 1 --2- 172.21.15.5:0/124758423 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c4c160820 0x7f8c4c0b6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.478+0000 7f8c52bbc640 1 --2- 172.21.15.5:0/124758423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c4c1603a0 0x7f8c4c0b5c30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.478+0000 7f8c52bbc640 1 --2- 172.21.15.5:0/124758423 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c38004d00 0x7f8c4c1524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:17:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.478+0000 7f8c52bbc640 1 -- 172.21.15.5:0/124758423 >> 172.21.15.5:0/124758423 conn(0x7f8c4c09c3e0 msgr2=0x7f8c4c06bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:17:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.479+0000 7f8c52bbc640 1 -- 172.21.15.5:0/124758423 shutdown_connections 2023-11-24T06:17:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:17:59.479+0000 7f8c52bbc640 1 -- 172.21.15.5:0/124758423 wait complete. 2023-11-24T06:17:59.500 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2023-11-24T06:17:58.126635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":241536,"kb_used_data":4548,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733184,"statfs":{"total":773094113280,"available":772846780416,"internally_reserved":0,"allocated":4657152,"data_stored":2552713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.126676"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":178,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-11-24T06:17:48.586416+0000","last_change":"2023-11-24T06:15:42.021608+0000","last_active":"2023-11-24T06:17:48.586416+0000","last_peered":"2023-11-24T06:17:48.586416+0000","last_clean":"2023-11-24T06:17:48.586416+0000","last_became_active":"2023-11-24T06:15:42.018847+0000","last_became_peered":"2023-11-24T06:15:42.018847+0000","last_unstale":"2023-11-24T06:17:48.586416+0000","last_undegraded":"2023-11-24T06:17:48.586416+0000","last_fullsized":"2023-11-24T06:17:48.586416+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T06:15:40.694368+0000","last_clean_scrub_stamp":"2023-11-24T06:15:40.694368+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:41.500663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32188,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339652,"statfs":{"total":96636764160,"available":96603803648,"internally_reserved":0,"allocated":1028096,"data_stored":761955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2370000000000001,"15min":1.2509999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.663,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.635},{"interface":"front","average":{"1min":1.238,"5min":1.2509999999999999,"15min":1.2529999999999999},"min":{"1min":0.70899999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7130000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.4299999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.2250000000000001,"15min":1.2330000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.3700000000000001,"15min":1.3879999999999999},"min":{"1min":0.85399999999999998,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6830000000000001,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":1.1000000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.4650000000000001,"15min":1.496},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.5980000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.486},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4930000000000001,"15min":1.51},"min":{"1min":0.86799999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.8280000000000001,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.0569999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.444,"15min":1.464},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.923,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":1.379},{"interface":"front","average":{"1min":1.476,"5min":1.6519999999999999,"15min":1.6819999999999999},"min":{"1min":0.92200000000000004,"5min":0.879,"15min":0.879},"max":{"1min":1.857,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":1.325}]},{"osd":4,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.774,"15min":1.8180000000000001},"min":{"1min":1.081,"5min":1.016,"15min":1.016},"max":{"1min":2.141,"5min":5.7560000000000002,"15min":5.7560000000000002},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.581,"5min":1.669,"15min":1.6839999999999999},"min":{"1min":1.26,"5min":1.018,"15min":1.018},"max":{"1min":2.0910000000000002,"5min":2.681,"15min":2.681},"last":1.167}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":2.0299999999999998,"15min":2.0880000000000001},"min":{"1min":1.0760000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.0960000000000001,"5min":6.6260000000000003,"15min":6.6260000000000003},"last":1.8029999999999999},{"interface":"front","average":{"1min":1.663,"5min":2.0750000000000002,"15min":2.1440000000000001},"min":{"1min":1.155,"5min":1.155,"15min":1.155},"max":{"1min":2.0459999999999998,"5min":6.4169999999999998,"15min":6.4169999999999998},"last":1.724}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.7749999999999999,"5min":1.972,"15min":2.0049999999999999},"min":{"1min":1.214,"5min":1.214,"15min":1.214},"max":{"1min":2.198,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":1.538},{"interface":"front","average":{"1min":1.633,"5min":1.853,"15min":1.889},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":2.173,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":1.292}]}]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.5449999999999999,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":0.82599999999999996},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.0980000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.569,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.96299999999999997}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.2370000000000001,"15min":1.246},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.77,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.266,"15min":1.2769999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.722,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.2070000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.246,"15min":1.2509999999999999},"min":{"1min":0.75700000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.994,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.161},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.4039999999999999,"15min":1.4199999999999999},"min":{"1min":0.84899999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.8029999999999999,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.113}]},{"osd":3,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.337,"15min":1.349},"min":{"1min":0.84099999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.796,"5min":2.331,"15min":2.331},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.53,"15min":1.55},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":1.3360000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.585,"15min":1.615},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.901,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.5229999999999999,"15min":1.5489999999999999},"min":{"1min":0.88600000000000001,"5min":0.873,"15min":0.873},"max":{"1min":1.9339999999999999,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.8120000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.6759999999999999,"15min":1.6930000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.976},{"interface":"front","average":{"1min":1.627,"5min":1.738,"15min":1.756},"min":{"1min":1.141,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":2.016,"5min":2.802,"15min":2.802},"last":1.7529999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.748,"15min":1.774},"min":{"1min":1.1399999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.048,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":1.927},{"interface":"front","average":{"1min":1.679,"5min":1.7549999999999999,"15min":1.7669999999999999},"min":{"1min":1.087,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.8680000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.1679999999999999,"15min":1.204},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.482,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.72599999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6040000000000001,"5min":1.875,"15min":1.875},"last":1.4039999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.169,"15min":1.175},"min":{"1min":0.79300000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.609,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.659},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.282,"15min":1.3200000000000001},"min":{"1min":0.69499999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.526,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.323}]},{"osd":2,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.147,"15min":1.167},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.343,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.4690000000000001,"15min":1.5149999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.6399999999999999,"5min":2.633,"15min":2.633},"last":1.1899999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.3660000000000001,"15min":1.3939999999999999},"min":{"1min":0.85499999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5249999999999999,"5min":2.323,"15min":2.323},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.544,"15min":1.5820000000000001},"min":{"1min":0.90500000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.694,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":1.4970000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.5620000000000001,"15min":1.5900000000000001},"min":{"1min":0.64500000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7569999999999999,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.7729999999999999},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.49,"15min":1.5189999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.8899999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":1.47}]},{"osd":5,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.635,"15min":1.663},"min":{"1min":0.83499999999999996,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.845,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.579,"15min":1.627},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6890000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.264}]},{"osd":7,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.738,"15min":1.7709999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.0230000000000001,"5min":2.6880000000000002,"15min":2.6880000000000002},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.6970000000000001,"15min":1.74},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.994,"5min":2.448,"15min":2.448},"last":1.4430000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0489999999999999,"15min":1.0369999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.131,"5min":1.1220000000000001,"15min":1.1200000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":0.85499999999999998}]},{"osd":1,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2529999999999999,"15min":1.2689999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.861,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.256,"15min":1.254},"min":{"1min":0.81100000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8200000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.2509999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.71899999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.55,"5min":1.677,"15min":1.677},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.359,"5min":1.419,"15min":1.429},"min":{"1min":0.72099999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.8520000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.3600000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2969999999999999,"15min":1.3029999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.655,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.222},{"interface":"front","average":{"1min":1.454,"5min":1.5169999999999999,"15min":1.5269999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9450000000000001,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":1.4710000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.556,"15min":1.5580000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.95,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.591},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.82799999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.1899999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":1.752}]},{"osd":6,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.575,"15min":1.571},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.298,"5min":2.6480000000000001,"15min":2.6480000000000001},"last":1.8700000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.6060000000000001,"15min":1.6220000000000001},"min":{"1min":1.028,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.887,"5min":3.1560000000000001,"15min":3.1560000000000001},"last":1.6180000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.655,"15min":1.657},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.23,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.7250000000000001},{"interface":"front","average":{"1min":1.6599999999999999,"5min":1.782,"15min":1.802},"min":{"1min":0.80100000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.1760000000000002,"5min":4.1779999999999999,"15min":4.1779999999999999},"last":1.6930000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31608,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.93400000000000005,"15min":0.95099999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2210000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.1040000000000001},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.03,"15min":1.044},"min":{"1min":0.48299999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.431,"5min":1.518,"15min":1.518},"last":1.153}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.236,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.083,"15min":1.1259999999999999},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0980000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":0.88800000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.151,"15min":1.1870000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.603,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.216},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.405,"5min":1.875,"15min":1.875},"last":0.68200000000000005}]},{"osd":4,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2310000000000001,"15min":1.2509999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.4770000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":0.96499999999999997},{"interface":"front","average":{"1min":1.202,"5min":1.4370000000000001,"15min":1.476},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.776,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.2729999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.3480000000000001,"15min":1.375},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5660000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.5189999999999999,"15min":1.552},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.865,"5min":3.0089999999999999,"15min":3.0089999999999999},"last":1.3280000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.4319999999999999,"15min":1.4490000000000001},"min":{"1min":0.96699999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.8360000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.25},{"interface":"front","average":{"1min":1.361,"5min":1.5,"15min":1.5229999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.944,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.1399999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.532,"15min":1.5589999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.9159999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.5780000000000001,"15min":1.609},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.869,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.02}]}]},{"osd":2,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31608,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.96099999999999997,"15min":0.94899999999999995},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.2709999999999999,"5min":5.2709999999999999,"15min":5.2709999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.079,"15min":1.0660000000000001},"min":{"1min":0.47499999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.5650000000000004,"5min":5.5650000000000004,"15min":5.5650000000000004},"last":1.02}]},{"osd":1,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1,"15min":0.99199999999999999},"min":{"1min":0.45700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":5.7140000000000004,"5min":5.7140000000000004,"15min":5.7140000000000004},"last":1.6399999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.0529999999999999,"15min":1.079},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.444,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":1.1499999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.131,"15min":1.1679999999999999},"min":{"1min":0.59599999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.403,"5min":1.95,"15min":1.95},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1839999999999999,"15min":1.1799999999999999},"min":{"1min":0.57699999999999996,"5min":0.504,"15min":0.504},"max":{"1min":5.4820000000000002,"5min":5.4820000000000002,"15min":5.4820000000000002},"last":1.262}]},{"osd":4,"last update":"Fri Nov 24 06:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3220000000000001,"15min":1.335},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.625,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.353},{"interface":"front","average":{"1min":1.171,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.944,"5min":2.944,"15min":2.944},"last":1.702}]},{"osd":5,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":4.1280000000000001,"5min":4.1280000000000001,"15min":4.1280000000000001},"last":1.407},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.4319999999999999,"15min":1.46},"min":{"1min":0.83099999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.7649999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.8440000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.427,"15min":1.4370000000000001},"min":{"1min":0.76700000000000002,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.6440000000000001,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.357,"15min":1.383},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.2120000000000002,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.5309999999999999}]},{"osd":7,"last update":"Fri Nov 24 06:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.45,"15min":1.4710000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.867,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.921},{"interface":"front","average":{"1min":1.55,"5min":1.5329999999999999,"15min":1.53},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":5.6630000000000003,"5min":5.6630000000000003,"15min":5.6630000000000003},"last":1.458}]}]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31608,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.89500000000000002,"15min":0.89100000000000001},"min":{"1min":0.66600000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.393,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.39800000000000002}]},{"osd":2,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.91100000000000003,"15min":0.90100000000000002},"min":{"1min":0.72399999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3340000000000001,"5min":1.919,"15min":1.919},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.177,"5min":1.121,"15min":1.1120000000000001},"min":{"1min":0.71599999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6719999999999999,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":0.76000000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.0940000000000001,"15min":1.077},"min":{"1min":0.56399999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5940000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.004,"15min":0.998},"min":{"1min":0.54900000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.746,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.3480000000000001,"15min":1.3340000000000001},"min":{"1min":1.2490000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7270000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3009999999999999,"15min":1.3049999999999999},"min":{"1min":0.90100000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.6579999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.94799999999999995}]},{"osd":5,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.304,"15min":1.2909999999999999},"min":{"1min":0.88200000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.982,"5min":2.085,"15min":2.085},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3959999999999999,"15min":1.397},"min":{"1min":0.94499999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.6279999999999999,"5min":2.355,"15min":2.355},"last":0.96799999999999997}]},{"osd":6,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.421,"15min":1.4159999999999999},"min":{"1min":1.046,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.944,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.468,"5min":1.4199999999999999,"15min":1.4119999999999999},"min":{"1min":0.96999999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8799999999999999,"5min":2.298,"15min":2.298},"last":0.92800000000000005}]},{"osd":7,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.544,"15min":1.5389999999999999},"min":{"1min":1.218,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.0299999999999998,"5min":2.492,"15min":2.492},"last":1.0129999999999999},{"interface":"front","average":{"1min":1.53,"5min":1.482,"15min":1.474},"min":{"1min":0.91400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.0830000000000002,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":0.90800000000000003}]}]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1024000,"data_stored":761236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.89300000000000002,"15min":0.91900000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.127,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.95499999999999996,"15min":0.97499999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5600000000000001,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.4850000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.91100000000000003,"15min":0.92200000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3680000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.048,"15min":1.0549999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.6699999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":1.167}]},{"osd":3,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.996,"15min":1.01},"min":{"1min":0.495,"5min":0.434,"15min":0.434},"max":{"1min":1.444,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.999,"5min":1.0980000000000001,"15min":1.1140000000000001},"min":{"1min":0.59099999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.522,"5min":2.653,"15min":2.653},"last":1.266}]},{"osd":4,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3320000000000001,"15min":1.347},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.8260000000000001,"5min":2.254,"15min":2.254},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3740000000000001,"15min":1.3939999999999999},"min":{"1min":0.84499999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8620000000000001,"5min":1.996,"15min":1.996},"last":1.421}]},{"osd":5,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.4259999999999999,"15min":1.4399999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.956,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.744},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.4410000000000001,"15min":1.4690000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.919,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.623}]},{"osd":6,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.5029999999999999,"15min":1.518},"min":{"1min":0.90200000000000002,"5min":0.746,"15min":0.746},"max":{"1min":2.0939999999999999,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.494,"15min":1.52},"min":{"1min":0.78100000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.891,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.8680000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.5429999999999999,"15min":1.5509999999999999},"min":{"1min":0.89200000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.228},{"interface":"front","average":{"1min":1.359,"5min":1.4990000000000001,"15min":1.522},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.925,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.7709999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:17:59.507 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-24T06:15:40.694368+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=6, tm_min=15, tm_sec=40, tm_wday=4, tm_yday=328, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=6, tm_min=17, tm_sec=52, tm_wday=4, tm_yday=328, tm_isdst=0) 2023-11-24T06:17:59.507 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-24T06:18:07.709 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:18:07.718 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:18:19.508 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T06:18:19.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.746+0000 7f4dcaebb640 1 Processor -- start 2023-11-24T06:18:19.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.747+0000 7f4dcaebb640 1 -- start start 2023-11-24T06:18:19.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.748+0000 7f4dcaebb640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f4dc40b1f00 0x7f4dc40d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:19.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.748+0000 7f4dcaebb640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc40afd30 con 0x7f4dc415d370 2023-11-24T06:18:19.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.748+0000 7f4dcaebb640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc40d0c30 con 0x7f4dc40d00d0 2023-11-24T06:18:19.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.748+0000 7f4dcaebb640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc40cef60 con 0x7f4dc40b1f00 2023-11-24T06:18:19.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.749+0000 7f4dc9eb9640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f4dc415d370 0x7f4dc40cf9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51728/0 (socket says 172.21.15.5:51728) 2023-11-24T06:18:19.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.749+0000 7f4dc9eb9640 1 -- 172.21.15.5:0/1372649756 learned_addr learned my addr 172.21.15.5:0/1372649756 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:18:19.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.749+0000 7f4dca6ba640 1 --2- 172.21.15.5:0/1372649756 >> v2:172.21.15.6:3300/0 conn(0x7f4dc40b1f00 0x7f4dc40d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:19.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.749+0000 7f4dc8eb7640 1 -- 172.21.15.5:0/1372649756 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3170997316 0 0) 0x7f4dc40afd30 con 0x7f4dc415d370 2023-11-24T06:18:19.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.749+0000 7f4dc8eb7640 1 -- 172.21.15.5:0/1372649756 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4db0003520 con 0x7f4dc415d370 2023-11-24T06:18:19.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.750+0000 7f4dc8eb7640 1 -- 172.21.15.5:0/1372649756 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 613819394 0 0) 0x7f4db0003520 con 0x7f4dc415d370 2023-11-24T06:18:19.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.750+0000 7f4dc8eb7640 1 -- 172.21.15.5:0/1372649756 >> v2:172.21.15.6:3300/0 conn(0x7f4dc40b1f00 msgr2=0x7f4dc40d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:18:19.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.750+0000 7f4dc8eb7640 1 --2- 172.21.15.5:0/1372649756 >> v2:172.21.15.6:3300/0 conn(0x7f4dc40b1f00 0x7f4dc40d1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:19.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.750+0000 7f4dc8eb7640 1 -- 172.21.15.5:0/1372649756 >> v1:172.21.15.6:6789/0 conn(0x7f4dc40d00d0 legacy=0x7f4dc40b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:19.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.750+0000 7f4dc8eb7640 1 -- 172.21.15.5:0/1372649756 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc40d0ef0 con 0x7f4dc415d370 2023-11-24T06:18:19.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.750+0000 7f4dc8eb7640 1 -- 172.21.15.5:0/1372649756 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f4dc0001670 con 0x7f4dc415d370 2023-11-24T06:18:19.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.750+0000 7f4dc8eb7640 1 -- 172.21.15.5:0/1372649756 >> v1:172.21.15.5:6789/0 conn(0x7f4dc415d370 legacy=0x7f4dc40cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.750+0000 7f4dc8eb7640 1 --2- 172.21.15.5:0/1372649756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4db0003a30 0x7f4db0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.750+0000 7f4dc8eb7640 1 --2- 172.21.15.5:0/1372649756 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4db00047c0 0x7f4db0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.750+0000 7f4dc8eb7640 1 --2- 172.21.15.5:0/1372649756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4db00052a0 0x7f4db000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.750+0000 7f4dc8eb7640 1 -- 172.21.15.5:0/1372649756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc40cef60 con 0x7f4db00052a0 2023-11-24T06:18:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.750+0000 7f4dc8eb7640 1 -- 172.21.15.5:0/1372649756 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db000c170 con 0x7f4db0003a30 2023-11-24T06:18:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.750+0000 7f4dc8eb7640 1 -- 172.21.15.5:0/1372649756 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db000c330 con 0x7f4db00047c0 2023-11-24T06:18:19.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.750+0000 7f4dca6ba640 1 --2- 172.21.15.5:0/1372649756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4db00052a0 0x7f4db000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.751+0000 7f4dc9eb9640 1 --2- 172.21.15.5:0/1372649756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4db0003a30 0x7f4db0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.751+0000 7f4dc96b8640 1 --2- 172.21.15.5:0/1372649756 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4db00047c0 0x7f4db0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.751+0000 7f4dca6ba640 1 -- 172.21.15.5:0/1372649756 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4db00047c0 msgr2=0x7f4db0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:18:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.751+0000 7f4dca6ba640 1 --2- 172.21.15.5:0/1372649756 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4db00047c0 0x7f4db0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.751+0000 7f4dca6ba640 1 -- 172.21.15.5:0/1372649756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4db0003a30 msgr2=0x7f4db0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:18:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.751+0000 7f4dca6ba640 1 --2- 172.21.15.5:0/1372649756 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4db0003a30 0x7f4db0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.751+0000 7f4dca6ba640 1 -- 172.21.15.5:0/1372649756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db0003520 con 0x7f4db00052a0 2023-11-24T06:18:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.752+0000 7f4dca6ba640 1 -- 172.21.15.5:0/1372649756 reap_dead start 2023-11-24T06:18:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.752+0000 7f4dca6ba640 1 --2- 172.21.15.5:0/1372649756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4db00052a0 0x7f4db000b980 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009fc0 tx=0x7f4db00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252529b0c11407eb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:18:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.752+0000 7f4dc8eb7640 1 -- 172.21.15.5:0/1372649756 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc02cc60 con 0x7f4db00052a0 2023-11-24T06:18:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.752+0000 7f4dc8eb7640 1 -- 172.21.15.5:0/1372649756 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dbc00f040 con 0x7f4db00052a0 2023-11-24T06:18:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.752+0000 7f4dc8eb7640 1 -- 172.21.15.5:0/1372649756 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc003990 con 0x7f4db00052a0 2023-11-24T06:18:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.753+0000 7f4dcaebb640 1 -- 172.21.15.5:0/1372649756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4db00052a0 msgr2=0x7f4db000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.753+0000 7f4dcaebb640 1 --2- 172.21.15.5:0/1372649756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4db00052a0 0x7f4db000b980 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009fc0 tx=0x7f4db00057f0 comp rx=0 tx=0).stop 2023-11-24T06:18:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.753+0000 7f4dcaebb640 1 -- 172.21.15.5:0/1372649756 shutdown_connections 2023-11-24T06:18:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.753+0000 7f4dcaebb640 1 --2- 172.21.15.5:0/1372649756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4db00052a0 0x7f4db000b980 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.753+0000 7f4dcaebb640 1 -- 172.21.15.5:0/1372649756 >> 172.21.15.5:0/1372649756 conn(0x7f4dc409c3d0 msgr2=0x7f4dc40d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:18:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.753+0000 7f4dcaebb640 1 -- 172.21.15.5:0/1372649756 shutdown_connections 2023-11-24T06:18:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.753+0000 7f4dcaebb640 1 -- 172.21.15.5:0/1372649756 wait complete. 2023-11-24T06:18:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.755+0000 7f4dcaebb640 1 Processor -- start 2023-11-24T06:18:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.755+0000 7f4dcaebb640 1 -- start start 2023-11-24T06:18:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.756+0000 7f4dcaebb640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4db00052a0 0x7f4dc41f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.756+0000 7f4dcaebb640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc414e310 0x7f4dc414e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.756+0000 7f4dcaebb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc414c280 0x7f4dc414c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.756+0000 7f4dcaebb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc414ecd0 con 0x7f4dc414c280 2023-11-24T06:18:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.756+0000 7f4dcaebb640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc414bff0 con 0x7f4dc414e310 2023-11-24T06:18:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.756+0000 7f4dcaebb640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc414cf40 con 0x7f4db00052a0 2023-11-24T06:18:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.756+0000 7f4dca6ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc414c280 0x7f4dc414c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.757+0000 7f4dca6ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc414c280 0x7f4dc414c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43090/0 (socket says 172.21.15.5:43090) 2023-11-24T06:18:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.757+0000 7f4dca6ba640 1 -- 172.21.15.5:0/3114790270 learned_addr learned my addr 172.21.15.5:0/3114790270 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:18:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.757+0000 7f4dc96b8640 1 --2- 172.21.15.5:0/3114790270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc414e310 0x7f4dc414e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.757+0000 7f4dc9eb9640 1 --2- 172.21.15.5:0/3114790270 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4db00052a0 0x7f4dc41f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.757+0000 7f4dca6ba640 1 -- 172.21.15.5:0/3114790270 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4db00052a0 msgr2=0x7f4dc41f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.757+0000 7f4dca6ba640 1 --2- 172.21.15.5:0/3114790270 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4db00052a0 0x7f4dc41f2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.757+0000 7f4dca6ba640 1 -- 172.21.15.5:0/3114790270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc414e310 msgr2=0x7f4dc414e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.757+0000 7f4dca6ba640 1 --2- 172.21.15.5:0/3114790270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc414e310 0x7f4dc414e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.757+0000 7f4dca6ba640 1 -- 172.21.15.5:0/3114790270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc414d220 con 0x7f4dc414c280 2023-11-24T06:18:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.758+0000 7f4dca6ba640 1 --2- 172.21.15.5:0/3114790270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc414c280 0x7f4dc414c700 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00e200 tx=0x7f4dbc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:18:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.758+0000 7f4dba7fc640 1 -- 172.21.15.5:0/3114790270 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc035e70 con 0x7f4dc414c280 2023-11-24T06:18:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.758+0000 7f4dba7fc640 1 -- 172.21.15.5:0/3114790270 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dbc003990 con 0x7f4dc414c280 2023-11-24T06:18:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.758+0000 7f4dcaebb640 1 -- 172.21.15.5:0/3114790270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc414d500 con 0x7f4dc414c280 2023-11-24T06:18:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.758+0000 7f4dba7fc640 1 -- 172.21.15.5:0/3114790270 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc003d40 con 0x7f4dc414c280 2023-11-24T06:18:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.758+0000 7f4dcaebb640 1 -- 172.21.15.5:0/3114790270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc414dac0 con 0x7f4dc414c280 2023-11-24T06:18:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.759+0000 7f4dcaebb640 1 -- 172.21.15.5:0/3114790270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc414dce0 con 0x7f4dc414c280 2023-11-24T06:18:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.760+0000 7f4dba7fc640 1 -- 172.21.15.5:0/3114790270 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f4dbc004030 con 0x7f4dc414c280 2023-11-24T06:18:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.762+0000 7f4dba7fc640 1 --2- 172.21.15.5:0/3114790270 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4da806af00 0x7f4da806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.762+0000 7f4dba7fc640 1 -- 172.21.15.5:0/3114790270 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4dbc0ac1b0 con 0x7f4dc414c280 2023-11-24T06:18:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.762+0000 7f4dc9eb9640 1 --2- 172.21.15.5:0/3114790270 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4da806af00 0x7f4da806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.763+0000 7f4dc9eb9640 1 --2- 172.21.15.5:0/3114790270 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4da806af00 0x7f4da806d3c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0001050 tx=0x7f4dc0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:18:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.766+0000 7f4dba7fc640 1 -- 172.21.15.5:0/3114790270 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4dbc077590 con 0x7f4dc414c280 2023-11-24T06:18:19.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.961+0000 7f4dcaebb640 1 -- 172.21.15.5:0/3114790270 --> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4dc415d750 con 0x7f4da806af00 2023-11-24T06:18:19.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.965+0000 7f4dba7fc640 1 -- 172.21.15.5:0/3114790270 <== mgr.4108 v2:172.21.15.5:6832/4230007794 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48427 (secure 0 0 0) 0x7f4dc0005e00 con 0x7f4da806af00 2023-11-24T06:18:19.968 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:18:19.969 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-11-24T06:18:19.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.967+0000 7f4dcaebb640 1 -- 172.21.15.5:0/3114790270 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4da806af00 msgr2=0x7f4da806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:19.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.967+0000 7f4dcaebb640 1 --2- 172.21.15.5:0/3114790270 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4da806af00 0x7f4da806d3c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0001050 tx=0x7f4dc0009040 comp rx=0 tx=0).stop 2023-11-24T06:18:19.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.967+0000 7f4dcaebb640 1 -- 172.21.15.5:0/3114790270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc414c280 msgr2=0x7f4dc414c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:19.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.967+0000 7f4dcaebb640 1 --2- 172.21.15.5:0/3114790270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc414c280 0x7f4dc414c700 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00e200 tx=0x7f4dbc009750 comp rx=0 tx=0).stop 2023-11-24T06:18:19.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.968+0000 7f4dcaebb640 1 -- 172.21.15.5:0/3114790270 shutdown_connections 2023-11-24T06:18:19.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.968+0000 7f4dcaebb640 1 --2- 172.21.15.5:0/3114790270 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f4da806af00 0x7f4da806d3c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.968+0000 7f4dcaebb640 1 --2- 172.21.15.5:0/3114790270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc414c280 0x7f4dc414c700 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.968+0000 7f4dcaebb640 1 --2- 172.21.15.5:0/3114790270 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f4dc414e310 0x7f4dc414e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.968+0000 7f4dcaebb640 1 --2- 172.21.15.5:0/3114790270 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f4db00052a0 0x7f4dc41f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.968+0000 7f4dcaebb640 1 -- 172.21.15.5:0/3114790270 >> 172.21.15.5:0/3114790270 conn(0x7f4dc409c3d0 msgr2=0x7f4dc40b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:18:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.968+0000 7f4dcaebb640 1 -- 172.21.15.5:0/3114790270 shutdown_connections 2023-11-24T06:18:19.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:19.968+0000 7f4dcaebb640 1 -- 172.21.15.5:0/3114790270 wait complete. 2023-11-24T06:18:19.990 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2023-11-24T06:18:18.132266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":241520,"kb_used_data":4548,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733200,"statfs":{"total":773094113280,"available":772846796800,"internally_reserved":0,"allocated":4657152,"data_stored":2552713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003456"},"pg_stats":[{"pgid":"1.0","version":"38'119","reported_seq":189,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-24T06:18:01.631548+0000","last_change":"2023-11-24T06:18:01.631618+0000","last_active":"2023-11-24T06:18:01.631548+0000","last_peered":"2023-11-24T06:18:01.631548+0000","last_clean":"2023-11-24T06:18:01.631548+0000","last_became_active":"2023-11-24T06:15:42.018847+0000","last_became_peered":"2023-11-24T06:15:42.018847+0000","last_unstale":"2023-11-24T06:18:01.631548+0000","last_undegraded":"2023-11-24T06:18:01.631548+0000","last_fullsized":"2023-11-24T06:18:01.631548+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"38'119","last_scrub_stamp":"2023-11-24T06:18:01.631369+0000","last_deep_scrub":"38'119","last_deep_scrub_stamp":"2023-11-24T06:18:01.631369+0000","last_clean_scrub_stamp":"2023-11-24T06:18:01.631369+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:09.626029+0000","scrub_duration":0.012714987000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1028096,"data_stored":761955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2370000000000001,"15min":1.2509999999999999},"min":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"max":{"1min":1.663,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.2509999999999999,"15min":1.2529999999999999},"min":{"1min":0.70899999999999996,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.7130000000000001,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.655}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1779999999999999,"5min":1.2250000000000001,"15min":1.2330000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.3700000000000001,"15min":1.3879999999999999},"min":{"1min":0.85399999999999998,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.6830000000000001,"5min":2.7719999999999998,"15min":2.7719999999999998},"last":1.478}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.4650000000000001,"15min":1.496},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.5980000000000001,"5min":2.8820000000000001,"15min":2.8820000000000001},"last":1.171},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4930000000000001,"15min":1.51},"min":{"1min":0.86799999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.8280000000000001,"5min":4.0579999999999998,"15min":4.0579999999999998},"last":1.1120000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.444,"15min":1.464},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.923,"5min":3.4239999999999999,"15min":3.4239999999999999},"last":1.008},{"interface":"front","average":{"1min":1.476,"5min":1.6519999999999999,"15min":1.6819999999999999},"min":{"1min":0.92200000000000004,"5min":0.879,"15min":0.879},"max":{"1min":1.857,"5min":3.0569999999999999,"15min":3.0569999999999999},"last":1.5660000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.774,"15min":1.8180000000000001},"min":{"1min":1.081,"5min":1.016,"15min":1.016},"max":{"1min":2.141,"5min":5.7560000000000002,"15min":5.7560000000000002},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.581,"5min":1.669,"15min":1.6839999999999999},"min":{"1min":1.26,"5min":1.018,"15min":1.018},"max":{"1min":2.0910000000000002,"5min":2.681,"15min":2.681},"last":1.3959999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.6859999999999999,"5min":2.0299999999999998,"15min":2.0880000000000001},"min":{"1min":1.0760000000000001,"5min":1.0009999999999999,"15min":1.0009999999999999},"max":{"1min":2.0960000000000001,"5min":6.6260000000000003,"15min":6.6260000000000003},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.663,"5min":2.0750000000000002,"15min":2.1440000000000001},"min":{"1min":1.155,"5min":1.155,"15min":1.155},"max":{"1min":2.0459999999999998,"5min":6.4169999999999998,"15min":6.4169999999999998},"last":1.359}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.7749999999999999,"5min":1.972,"15min":2.0049999999999999},"min":{"1min":1.214,"5min":1.214,"15min":1.214},"max":{"1min":2.198,"5min":4.4050000000000002,"15min":4.4050000000000002},"last":1.446},{"interface":"front","average":{"1min":1.633,"5min":1.853,"15min":1.889},"min":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"max":{"1min":2.173,"5min":3.8540000000000001,"15min":3.8540000000000001},"last":1.6000000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.103,"5min":1.1259999999999999,"15min":1.1299999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.5449999999999999,"5min":2.3210000000000002,"15min":2.3210000000000002},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.0980000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.569,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.3380000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.2370000000000001,"15min":1.246},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.77,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.266,"15min":1.2769999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.722,"5min":2.8420000000000001,"15min":2.8420000000000001},"last":1.71}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.246,"15min":1.2509999999999999},"min":{"1min":0.75700000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.994,"5min":2.0219999999999998,"15min":2.0219999999999998},"last":1.135},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.4039999999999999,"15min":1.4199999999999999},"min":{"1min":0.84899999999999998,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.8029999999999999,"5min":1.9650000000000001,"15min":1.9650000000000001},"last":1.6519999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.337,"15min":1.349},"min":{"1min":0.84099999999999997,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.796,"5min":2.331,"15min":2.331},"last":1.056},{"interface":"front","average":{"1min":1.407,"5min":1.53,"15min":1.55},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":1.385}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.585,"15min":1.615},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.901,"5min":2.6989999999999998,"15min":2.6989999999999998},"last":1.23},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.5229999999999999,"15min":1.5489999999999999},"min":{"1min":0.88600000000000001,"5min":0.873,"15min":0.873},"max":{"1min":1.9339999999999999,"5min":2.4740000000000002,"15min":2.4740000000000002},"last":1.1799999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.573,"5min":1.6759999999999999,"15min":1.6930000000000001},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.8290000000000002,"5min":2.8290000000000002,"15min":2.8290000000000002},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.627,"5min":1.738,"15min":1.756},"min":{"1min":1.141,"5min":0.93500000000000005,"15min":0.93500000000000005},"max":{"1min":2.016,"5min":2.802,"15min":2.802},"last":1.5840000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.748,"15min":1.774},"min":{"1min":1.1399999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.048,"5min":3.4550000000000001,"15min":3.4550000000000001},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.679,"5min":1.7549999999999999,"15min":1.7669999999999999},"min":{"1min":1.087,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.4890000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542177,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":1.1679999999999999,"15min":1.204},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.482,"5min":1.7410000000000001,"15min":1.7410000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.151,"15min":1.159},"min":{"1min":0.72599999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.6040000000000001,"5min":1.875,"15min":1.875},"last":1.036}]},{"osd":1,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.169,"15min":1.175},"min":{"1min":0.79300000000000004,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.609,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.107},{"interface":"front","average":{"1min":1.0580000000000001,"5min":1.282,"15min":1.3200000000000001},"min":{"1min":0.69499999999999995,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.526,"5min":2.1110000000000002,"15min":2.1110000000000002},"last":1.2909999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.147,"15min":1.167},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.343,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.4690000000000001,"15min":1.5149999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.6399999999999999,"5min":2.633,"15min":2.633},"last":1.3420000000000001}]},{"osd":3,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.3660000000000001,"15min":1.3939999999999999},"min":{"1min":0.85499999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5249999999999999,"5min":2.323,"15min":2.323},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.544,"15min":1.5820000000000001},"min":{"1min":0.90500000000000003,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.694,"5min":3.5249999999999999,"15min":3.5249999999999999},"last":1.4950000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.5620000000000001,"15min":1.5900000000000001},"min":{"1min":0.64500000000000002,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.7569999999999999,"5min":2.7290000000000001,"15min":2.7290000000000001},"last":1.6160000000000001},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.49,"15min":1.5189999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.8899999999999999,"5min":2.3149999999999999,"15min":2.3149999999999999},"last":0.91400000000000003}]},{"osd":5,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.635,"15min":1.663},"min":{"1min":0.83499999999999996,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.845,"5min":2.5179999999999998,"15min":2.5179999999999998},"last":1.167},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.579,"15min":1.627},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6890000000000001,"5min":3.1459999999999999,"15min":3.1459999999999999},"last":1.385}]},{"osd":7,"last update":"Fri Nov 24 06:17:41 2023","interfaces":[{"interface":"back","average":{"1min":1.5409999999999999,"5min":1.738,"15min":1.7709999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.0230000000000001,"5min":2.6880000000000002,"15min":2.6880000000000002},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.6970000000000001,"15min":1.74},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.994,"5min":2.448,"15min":2.448},"last":1.556}]}]},{"osd":5,"up_from":14,"seq":60129542179,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27448,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344392,"statfs":{"total":96636764160,"available":96608657408,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.0489999999999999,"15min":1.0369999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.6870000000000001},"last":1.091},{"interface":"front","average":{"1min":1.131,"5min":1.1220000000000001,"15min":1.1200000000000001},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":2.0009999999999999,"5min":2.0009999999999999,"15min":2.0009999999999999},"last":1.4970000000000001}]},{"osd":1,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2529999999999999,"15min":1.2689999999999999},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.861,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.256,"15min":1.254},"min":{"1min":0.81100000000000005,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.8200000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.157}]},{"osd":2,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.71899999999999997,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.55,"5min":1.677,"15min":1.677},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.419,"15min":1.429},"min":{"1min":0.72099999999999997,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.8520000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.2949999999999999}]},{"osd":3,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2969999999999999,"15min":1.3029999999999999},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"max":{"1min":1.655,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.679},{"interface":"front","average":{"1min":1.454,"5min":1.5169999999999999,"15min":1.5269999999999999},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9450000000000001,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":1.4470000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.556,"15min":1.5580000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.95,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.804},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.82799999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.1899999999999999,"5min":2.5089999999999999,"15min":2.5089999999999999},"last":1.268}]},{"osd":6,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.597,"5min":1.575,"15min":1.571},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.298,"5min":2.6480000000000001,"15min":2.6480000000000001},"last":1.383},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.6060000000000001,"15min":1.6220000000000001},"min":{"1min":1.028,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.887,"5min":3.1560000000000001,"15min":3.1560000000000001},"last":1.3440000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:42 2023","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.655,"15min":1.657},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.23,"5min":3.2949999999999999,"15min":3.2949999999999999},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.6599999999999999,"5min":1.782,"15min":1.802},"min":{"1min":0.80100000000000005,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.1760000000000002,"5min":4.1779999999999999,"15min":4.1779999999999999},"last":1.6020000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542177,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31608,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.93400000000000005,"15min":0.95099999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.2210000000000001,"5min":1.6699999999999999,"15min":1.6699999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":1.03,"15min":1.044},"min":{"1min":0.48299999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.431,"5min":1.518,"15min":1.518},"last":1.208}]},{"osd":1,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.92000000000000004,"15min":0.93999999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.236,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":1.083,"15min":1.1259999999999999},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0980000000000001,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.7649999999999999}]},{"osd":2,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.151,"15min":1.1870000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.603,"5min":2.5289999999999999,"15min":2.5289999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.0309999999999999,"15min":1.046},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.405,"5min":1.875,"15min":1.875},"last":1.3779999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.2310000000000001,"15min":1.2509999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":1.4770000000000001,"5min":2.1200000000000001,"15min":2.1200000000000001},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.202,"5min":1.4370000000000001,"15min":1.476},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.776,"5min":2.8279999999999998,"15min":2.8279999999999998},"last":1.6419999999999999}]},{"osd":5,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.3480000000000001,"15min":1.375},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.5660000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.153},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.5189999999999999,"15min":1.552},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.865,"5min":3.0089999999999999,"15min":3.0089999999999999},"last":1.901}]},{"osd":6,"last update":"Fri Nov 24 06:17:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.4319999999999999,"15min":1.4490000000000001},"min":{"1min":0.96699999999999997,"5min":0.91100000000000003,"15min":0.91100000000000003},"max":{"1min":1.8360000000000001,"5min":2.3500000000000001,"15min":2.3500000000000001},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.361,"5min":1.5,"15min":1.5229999999999999},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.944,"5min":2.7370000000000001,"15min":2.7370000000000001},"last":1.6020000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3700000000000001,"5min":1.532,"15min":1.5589999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.9159999999999999,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.5780000000000001,"15min":1.609},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.869,"5min":3.1280000000000001,"15min":3.1280000000000001},"last":1.827}]}]},{"osd":2,"up_from":14,"seq":60129542178,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31608,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":0.96099999999999997,"15min":0.94899999999999995},"min":{"1min":0.38500000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":5.2709999999999999,"5min":5.2709999999999999,"15min":5.2709999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.157,"5min":1.079,"15min":1.0660000000000001},"min":{"1min":0.47499999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.5650000000000004,"5min":5.5650000000000004,"15min":5.5650000000000004},"last":1.0049999999999999}]},{"osd":1,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1,"15min":0.99199999999999999},"min":{"1min":0.45700000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":5.7140000000000004,"5min":5.7140000000000004,"15min":5.7140000000000004},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.89700000000000002,"5min":1.0529999999999999,"15min":1.079},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.444,"5min":2.6259999999999999,"15min":2.6259999999999999},"last":1.504}]},{"osd":3,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":1.131,"15min":1.1679999999999999},"min":{"1min":0.59599999999999997,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.403,"5min":1.95,"15min":1.95},"last":1.056},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1839999999999999,"15min":1.1799999999999999},"min":{"1min":0.57699999999999996,"5min":0.504,"15min":0.504},"max":{"1min":5.4820000000000002,"5min":5.4820000000000002,"15min":5.4820000000000002},"last":1.1479999999999999}]},{"osd":4,"last update":"Fri Nov 24 06:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3220000000000001,"15min":1.335},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.625,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":1.393},{"interface":"front","average":{"1min":1.171,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.944,"5min":2.944,"15min":2.944},"last":1.4330000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2869999999999999,"15min":1.2969999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":4.1280000000000001,"5min":4.1280000000000001,"15min":4.1280000000000001},"last":1.738},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.4319999999999999,"15min":1.46},"min":{"1min":0.83099999999999996,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.7649999999999999,"5min":2.0350000000000001,"15min":2.0350000000000001},"last":1.2370000000000001}]},{"osd":6,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.427,"15min":1.4370000000000001},"min":{"1min":0.76700000000000002,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.6440000000000001,"5min":3.0499999999999998,"15min":3.0499999999999998},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.357,"15min":1.383},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"max":{"1min":2.2120000000000002,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":1.615}]},{"osd":7,"last update":"Fri Nov 24 06:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.45,"15min":1.4710000000000001},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.867,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.5329999999999999,"15min":1.53},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":5.6630000000000003,"5min":5.6630000000000003,"15min":5.6630000000000003},"last":1.675}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31608,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340232,"statfs":{"total":96636764160,"available":96604397568,"internally_reserved":0,"allocated":434176,"data_stored":171587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.78600000000000003,"15min":0.76400000000000001},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.89500000000000002,"15min":0.89100000000000001},"min":{"1min":0.66600000000000004,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.393,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.82799999999999996}]},{"osd":2,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.91100000000000003,"15min":0.90100000000000002},"min":{"1min":0.72399999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3340000000000001,"5min":1.919,"15min":1.919},"last":1.165},{"interface":"front","average":{"1min":1.177,"5min":1.121,"15min":1.1120000000000001},"min":{"1min":0.71599999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.6719999999999999,"5min":2.3090000000000002,"15min":2.3090000000000002},"last":1.464}]},{"osd":3,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.0940000000000001,"15min":1.077},"min":{"1min":0.56399999999999995,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.5940000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.004,"15min":0.998},"min":{"1min":0.54900000000000004,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.746,"5min":2.5619999999999998,"15min":2.5619999999999998},"last":1.0980000000000001}]},{"osd":4,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.3480000000000001,"15min":1.3340000000000001},"min":{"1min":1.2490000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.7270000000000001,"5min":2.1659999999999999,"15min":2.1659999999999999},"last":1.405},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3009999999999999,"15min":1.3049999999999999},"min":{"1min":0.90100000000000002,"5min":0.748,"15min":0.748},"max":{"1min":1.6579999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.379}]},{"osd":5,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.304,"15min":1.2909999999999999},"min":{"1min":0.88200000000000001,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.982,"5min":2.085,"15min":2.085},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3959999999999999,"15min":1.397},"min":{"1min":0.94499999999999995,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.6279999999999999,"5min":2.355,"15min":2.355},"last":1.704}]},{"osd":6,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.421,"15min":1.4159999999999999},"min":{"1min":1.046,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.944,"5min":2.3050000000000002,"15min":2.3050000000000002},"last":1.23},{"interface":"front","average":{"1min":1.468,"5min":1.4199999999999999,"15min":1.4119999999999999},"min":{"1min":0.96999999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.8799999999999999,"5min":2.298,"15min":2.298},"last":1.5920000000000001}]},{"osd":7,"last update":"Fri Nov 24 06:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.544,"15min":1.5389999999999999},"min":{"1min":1.218,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.0299999999999998,"5min":2.492,"15min":2.492},"last":1.78},{"interface":"front","average":{"1min":1.53,"5min":1.482,"15min":1.474},"min":{"1min":0.91400000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":2.0830000000000002,"5min":2.4079999999999999,"15min":2.4079999999999999},"last":1.635}]}]},{"osd":0,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1024000,"data_stored":761236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.89300000000000002,"15min":0.91900000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.127,"5min":2.7320000000000002,"15min":2.7320000000000002},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.95499999999999996,"15min":0.97499999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.5600000000000001,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.3120000000000001}]},{"osd":2,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.91100000000000003,"15min":0.92200000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3680000000000001,"5min":2.4870000000000001,"15min":2.4870000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.048,"15min":1.0549999999999999},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.6699999999999999,"5min":2.5209999999999999,"15min":2.5209999999999999},"last":0.90400000000000003}]},{"osd":3,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.996,"15min":1.01},"min":{"1min":0.495,"5min":0.434,"15min":0.434},"max":{"1min":1.444,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":1.359},{"interface":"front","average":{"1min":0.999,"5min":1.0980000000000001,"15min":1.1140000000000001},"min":{"1min":0.59099999999999997,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.522,"5min":2.653,"15min":2.653},"last":0.96199999999999997}]},{"osd":4,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3320000000000001,"15min":1.347},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.8260000000000001,"5min":2.254,"15min":2.254},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3740000000000001,"15min":1.3939999999999999},"min":{"1min":0.84499999999999997,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8620000000000001,"5min":1.996,"15min":1.996},"last":1.4810000000000001}]},{"osd":5,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.4259999999999999,"15min":1.4399999999999999},"min":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.956,"5min":2.4380000000000002,"15min":2.4380000000000002},"last":1.284},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.4410000000000001,"15min":1.4690000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":1.919,"5min":2.3799999999999999,"15min":2.3799999999999999},"last":1.6699999999999999}]},{"osd":6,"last update":"Fri Nov 24 06:17:40 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.5029999999999999,"15min":1.518},"min":{"1min":0.90200000000000002,"5min":0.746,"15min":0.746},"max":{"1min":2.0939999999999999,"5min":2.3159999999999998,"15min":2.3159999999999998},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.494,"15min":1.52},"min":{"1min":0.78100000000000003,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.891,"5min":2.5699999999999998,"15min":2.5699999999999998},"last":1.179}]},{"osd":7,"last update":"Fri Nov 24 06:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.5429999999999999,"15min":1.5509999999999999},"min":{"1min":0.89200000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.4990000000000001,"15min":1.522},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.925,"5min":2.6150000000000002,"15min":2.6150000000000002},"last":1.423}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T06:18:19.992 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-24T06:18:20.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.207+0000 7f893e4e0640 1 Processor -- start 2023-11-24T06:18:20.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.207+0000 7f893e4e0640 1 -- start start 2023-11-24T06:18:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.209+0000 7f893e4e0640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8938070a60 0x7f89380cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.209+0000 7f893e4e0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f89380cd1e0 0x7f89380cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:20.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.209+0000 7f893e4e0640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89380cbd70 con 0x7f893815b120 2023-11-24T06:18:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.209+0000 7f893e4e0640 1 -- --> v2:172.21.15.6:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89380cb7b0 con 0x7f8938070a60 2023-11-24T06:18:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.209+0000 7f893e4e0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89380cba90 con 0x7f89380cd1e0 2023-11-24T06:18:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.209+0000 7f89377fe640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8938070a60 0x7f89380cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.210+0000 7f893ca7d640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f89380cd1e0 0x7f89380cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.210+0000 7f89377fe640 1 --2- >> v2:172.21.15.6:3300/0 conn(0x7f8938070a60 0x7f89380cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.6:3300/0 says I am v2:172.21.15.5:49380/0 (socket says 172.21.15.5:49380) 2023-11-24T06:18:20.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.210+0000 7f89377fe640 1 -- 172.21.15.5:0/2790374073 learned_addr learned my addr 172.21.15.5:0/2790374073 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:18:20.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.210+0000 7f89377fe640 1 -- 172.21.15.5:0/2790374073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f89380cd1e0 msgr2=0x7f89380cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:18:20.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.210+0000 7f89377fe640 1 --2- 172.21.15.5:0/2790374073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f89380cd1e0 0x7f89380cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.210+0000 7f89377fe640 1 -- 172.21.15.5:0/2790374073 >> v1:172.21.15.6:6789/0 conn(0x7f893815b120 legacy=0x7f89380cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:20.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.210+0000 7f89377fe640 1 -- 172.21.15.5:0/2790374073 --> v2:172.21.15.6:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89380d0840 con 0x7f8938070a60 2023-11-24T06:18:20.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.212+0000 7f89377fe640 1 --2- 172.21.15.5:0/2790374073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8938070a60 0x7f89380cf310 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f892c009870 tx=0x7f892c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2a15ccbd5244d74 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:18:20.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.213+0000 7f8936ffd640 1 -- 172.21.15.5:0/2790374073 <== mon.1 v2:172.21.15.6:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c031640 con 0x7f8938070a60 2023-11-24T06:18:20.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.213+0000 7f8936ffd640 1 -- 172.21.15.5:0/2790374073 <== mon.1 v2:172.21.15.6:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892c0317a0 con 0x7f8938070a60 2023-11-24T06:18:20.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.213+0000 7f8936ffd640 1 -- 172.21.15.5:0/2790374073 <== mon.1 v2:172.21.15.6:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892c031b30 con 0x7f8938070a60 2023-11-24T06:18:20.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.213+0000 7f893e4e0640 1 -- 172.21.15.5:0/2790374073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8938070a60 msgr2=0x7f89380cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:20.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.213+0000 7f893e4e0640 1 --2- 172.21.15.5:0/2790374073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8938070a60 0x7f89380cf310 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f892c009870 tx=0x7f892c02b8f0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.214+0000 7f893e4e0640 1 -- 172.21.15.5:0/2790374073 shutdown_connections 2023-11-24T06:18:20.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.214+0000 7f893e4e0640 1 --2- 172.21.15.5:0/2790374073 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f89380cd1e0 0x7f89380cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.214+0000 7f893e4e0640 1 --2- 172.21.15.5:0/2790374073 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8938070a60 0x7f89380cf310 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.214+0000 7f893e4e0640 1 -- 172.21.15.5:0/2790374073 >> 172.21.15.5:0/2790374073 conn(0x7f89380b2ba0 msgr2=0x7f89380b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:18:20.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.214+0000 7f893e4e0640 1 -- 172.21.15.5:0/2790374073 shutdown_connections 2023-11-24T06:18:20.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.214+0000 7f893e4e0640 1 -- 172.21.15.5:0/2790374073 wait complete. 2023-11-24T06:18:20.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.216+0000 7f893e4e0640 1 Processor -- start 2023-11-24T06:18:20.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.216+0000 7f893e4e0640 1 -- start start 2023-11-24T06:18:20.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.217+0000 7f893e4e0640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8938070a60 0x7f89380c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:20.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.217+0000 7f893e4e0640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89380cd1e0 0x7f89380caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:20.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.217+0000 7f893e4e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893815b120 0x7f8938157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.217+0000 7f893e4e0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89380c74f0 con 0x7f893815b120 2023-11-24T06:18:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.217+0000 7f893e4e0640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89380cb0e0 con 0x7f89380cd1e0 2023-11-24T06:18:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.217+0000 7f893e4e0640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8938158520 con 0x7f8938070a60 2023-11-24T06:18:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.217+0000 7f893ca7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893815b120 0x7f8938157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.217+0000 7f893ca7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893815b120 0x7f8938157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43092/0 (socket says 172.21.15.5:43092) 2023-11-24T06:18:20.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.217+0000 7f893ca7d640 1 -- 172.21.15.5:0/4070249578 learned_addr learned my addr 172.21.15.5:0/4070249578 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:18:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.218+0000 7f89377fe640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89380cd1e0 0x7f89380caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.218+0000 7f8937fff640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8938070a60 0x7f89380c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.218+0000 7f893ca7d640 1 -- 172.21.15.5:0/4070249578 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8938070a60 msgr2=0x7f89380c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.218+0000 7f893ca7d640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8938070a60 0x7f89380c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.218+0000 7f893ca7d640 1 -- 172.21.15.5:0/4070249578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89380cd1e0 msgr2=0x7f89380caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:20.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.218+0000 7f893ca7d640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89380cd1e0 0x7f89380caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.218+0000 7f893ca7d640 1 -- 172.21.15.5:0/4070249578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c0094d0 con 0x7f893815b120 2023-11-24T06:18:20.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.218+0000 7f89377fe640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89380cd1e0 0x7f89380caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T06:18:20.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.218+0000 7f893ca7d640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893815b120 0x7f8938157fe0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f893000efc0 tx=0x7f893000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:18:20.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.219+0000 7f8934ff9640 1 -- 172.21.15.5:0/4070249578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930014480 con 0x7f893815b120 2023-11-24T06:18:20.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.219+0000 7f8934ff9640 1 -- 172.21.15.5:0/4070249578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893000f040 con 0x7f893815b120 2023-11-24T06:18:20.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.219+0000 7f893e4e0640 1 -- 172.21.15.5:0/4070249578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8938159320 con 0x7f893815b120 2023-11-24T06:18:20.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.219+0000 7f8934ff9640 1 -- 172.21.15.5:0/4070249578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930014820 con 0x7f893815b120 2023-11-24T06:18:20.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.219+0000 7f893e4e0640 1 -- 172.21.15.5:0/4070249578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8938158a80 con 0x7f893815b120 2023-11-24T06:18:20.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.220+0000 7f893e4e0640 1 -- 172.21.15.5:0/4070249578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893806b9c0 con 0x7f893815b120 2023-11-24T06:18:20.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.221+0000 7f8934ff9640 1 -- 172.21.15.5:0/4070249578 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8930014980 con 0x7f893815b120 2023-11-24T06:18:20.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.222+0000 7f8934ff9640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f892406aed0 0x7f892406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:20.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.222+0000 7f8934ff9640 1 -- 172.21.15.5:0/4070249578 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f893008a180 con 0x7f893815b120 2023-11-24T06:18:20.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.222+0000 7f8937fff640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f892406aed0 0x7f892406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:20.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.223+0000 7f8937fff640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f892406aed0 0x7f892406d390 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f893806b970 tx=0x7f8928009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:18:20.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.230+0000 7f8934ff9640 1 -- 172.21.15.5:0/4070249578 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f893806b9c0 con 0x7f893815b120 2023-11-24T06:18:20.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.430+0000 7f893e4e0640 1 -- 172.21.15.5:0/4070249578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f893806b9c0 con 0x7f893815b120 2023-11-24T06:18:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.436+0000 7f8934ff9640 1 -- 172.21.15.5:0/4070249578 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f893806b9c0 con 0x7f893815b120 2023-11-24T06:18:20.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.439+0000 7f8934ff9640 1 -- 172.21.15.5:0/4070249578 <== mon.0 v2:172.21.15.5:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8930055330 con 0x7f893815b120 2023-11-24T06:18:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.441+0000 7f893e4e0640 1 -- 172.21.15.5:0/4070249578 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f892406aed0 msgr2=0x7f892406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.441+0000 7f893e4e0640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f892406aed0 0x7f892406d390 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f893806b970 tx=0x7f8928009040 comp rx=0 tx=0).stop 2023-11-24T06:18:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.441+0000 7f893e4e0640 1 -- 172.21.15.5:0/4070249578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893815b120 msgr2=0x7f8938157fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.441+0000 7f893e4e0640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893815b120 0x7f8938157fe0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f893000efc0 tx=0x7f893000c490 comp rx=0 tx=0).stop 2023-11-24T06:18:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.441+0000 7f893e4e0640 1 -- 172.21.15.5:0/4070249578 shutdown_connections 2023-11-24T06:18:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.441+0000 7f893e4e0640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f892406aed0 0x7f892406d390 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.441+0000 7f893e4e0640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893815b120 0x7f8938157fe0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.441+0000 7f893e4e0640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f89380cd1e0 0x7f89380caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.441+0000 7f893e4e0640 1 --2- 172.21.15.5:0/4070249578 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8938070a60 0x7f89380c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.441+0000 7f893e4e0640 1 -- 172.21.15.5:0/4070249578 >> 172.21.15.5:0/4070249578 conn(0x7f89380b2ba0 msgr2=0x7f89380b4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:18:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.441+0000 7f893e4e0640 1 -- 172.21.15.5:0/4070249578 shutdown_connections 2023-11-24T06:18:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.441+0000 7f893e4e0640 1 -- 172.21.15.5:0/4070249578 wait complete. 2023-11-24T06:18:20.464 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T06:18:20.464 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T06:18:20.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.682+0000 7f8c9e41c640 1 Processor -- start 2023-11-24T06:18:20.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.683+0000 7f8c9e41c640 1 -- start start 2023-11-24T06:18:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.684+0000 7f8c9e41c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c98070780 0x7f8c9806d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.684+0000 7f8c9e41c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c98071cd0 con 0x7f8c98070cd0 2023-11-24T06:18:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.684+0000 7f8c9e41c640 1 -- --> v1:172.21.15.6:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c980b2120 con 0x7f8c98160970 2023-11-24T06:18:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.684+0000 7f8c9e41c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c980b2400 con 0x7f8c98070780 2023-11-24T06:18:20.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.685+0000 7f8c9c9b9640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8c98070cd0 0x7f8c980acbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51732/0 (socket says 172.21.15.5:51732) 2023-11-24T06:18:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.685+0000 7f8c9c9b9640 1 -- 172.21.15.5:0/3595681659 learned_addr learned my addr 172.21.15.5:0/3595681659 (peer_addr_for_me v1:172.21.15.5:0/0) 2023-11-24T06:18:20.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.685+0000 7f8c8ffff640 1 --2- 172.21.15.5:0/3595681659 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c98070780 0x7f8c9806d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:20.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.685+0000 7f8c977fe640 1 -- 172.21.15.5:0/3595681659 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361367598 0 0) 0x7f8c98071cd0 con 0x7f8c98070cd0 2023-11-24T06:18:20.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.685+0000 7f8c977fe640 1 -- 172.21.15.5:0/3595681659 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c84003520 con 0x7f8c98070cd0 2023-11-24T06:18:20.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.686+0000 7f8c977fe640 1 -- 172.21.15.5:0/3595681659 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1649098994 0 0) 0x7f8c84003520 con 0x7f8c98070cd0 2023-11-24T06:18:20.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.686+0000 7f8c977fe640 1 -- 172.21.15.5:0/3595681659 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c98070780 msgr2=0x7f8c9806d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:18:20.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.686+0000 7f8c977fe640 1 --2- 172.21.15.5:0/3595681659 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c98070780 0x7f8c9806d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.686+0000 7f8c977fe640 1 -- 172.21.15.5:0/3595681659 >> v1:172.21.15.6:6789/0 conn(0x7f8c98160970 legacy=0x7f8c980b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:20.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.686+0000 7f8c977fe640 1 -- 172.21.15.5:0/3595681659 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c980b26e0 con 0x7f8c98070cd0 2023-11-24T06:18:20.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.686+0000 7f8c977fe640 1 -- 172.21.15.5:0/3595681659 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1385296611 0 0) 0x7f8c900019f0 con 0x7f8c98070cd0 2023-11-24T06:18:20.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.686+0000 7f8c977fe640 1 -- 172.21.15.5:0/3595681659 >> v1:172.21.15.5:6789/0 conn(0x7f8c98070cd0 legacy=0x7f8c980acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:20.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.686+0000 7f8c977fe640 1 --2- 172.21.15.5:0/3595681659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c840039a0 0x7f8c84003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:20.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.686+0000 7f8c977fe640 1 --2- 172.21.15.5:0/3595681659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c840047c0 0x7f8c84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:20.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.686+0000 7f8c977fe640 1 --2- 172.21.15.5:0/3595681659 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c98070780 0x7f8c840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.686+0000 7f8c977fe640 1 -- 172.21.15.5:0/3595681659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c980b2400 con 0x7f8c840047c0 2023-11-24T06:18:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.686+0000 7f8c977fe640 1 -- 172.21.15.5:0/3595681659 --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8400bab0 con 0x7f8c840039a0 2023-11-24T06:18:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.686+0000 7f8c977fe640 1 -- 172.21.15.5:0/3595681659 --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8400bc70 con 0x7f8c98070780 2023-11-24T06:18:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.686+0000 7f8c8ffff640 1 --2- 172.21.15.5:0/3595681659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c840047c0 0x7f8c84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:20.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.687+0000 7f8c8ffff640 1 -- 172.21.15.5:0/3595681659 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c98070780 msgr2=0x7f8c840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:18:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.687+0000 7f8c8ffff640 1 --2- 172.21.15.5:0/3595681659 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c98070780 0x7f8c840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.687+0000 7f8c97fff640 1 --2- 172.21.15.5:0/3595681659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c840039a0 0x7f8c84003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.687+0000 7f8c8ffff640 1 -- 172.21.15.5:0/3595681659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c840039a0 msgr2=0x7f8c84003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T06:18:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.687+0000 7f8c8ffff640 1 --2- 172.21.15.5:0/3595681659 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c840039a0 0x7f8c84003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.687+0000 7f8c8ffff640 1 -- 172.21.15.5:0/3595681659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c84003520 con 0x7f8c840047c0 2023-11-24T06:18:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.688+0000 7f8c8ffff640 1 --2- 172.21.15.5:0/3595681659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c840047c0 0x7f8c84004c40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009ac0 tx=0x7f8c840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4c8ea8cf24155c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:18:20.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.688+0000 7f8c977fe640 1 -- 172.21.15.5:0/3595681659 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8002c710 con 0x7f8c840047c0 2023-11-24T06:18:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.688+0000 7f8c977fe640 1 -- 172.21.15.5:0/3595681659 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8c8002c870 con 0x7f8c840047c0 2023-11-24T06:18:20.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.688+0000 7f8c977fe640 1 -- 172.21.15.5:0/3595681659 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8002cc20 con 0x7f8c840047c0 2023-11-24T06:18:20.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.689+0000 7f8c9e41c640 1 -- 172.21.15.5:0/3595681659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c840047c0 msgr2=0x7f8c84004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:20.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.689+0000 7f8c9e41c640 1 --2- 172.21.15.5:0/3595681659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c840047c0 0x7f8c84004c40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009ac0 tx=0x7f8c840057f0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.689+0000 7f8c9c9b9640 1 -- 172.21.15.5:0/3595681659 reap_dead start 2023-11-24T06:18:20.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.689+0000 7f8c9e41c640 1 -- 172.21.15.5:0/3595681659 shutdown_connections 2023-11-24T06:18:20.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.689+0000 7f8c9e41c640 1 -- 172.21.15.5:0/3595681659 >> 172.21.15.5:0/3595681659 conn(0x7f8c980b0d80 msgr2=0x7f8c980b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:18:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.689+0000 7f8c9e41c640 1 -- 172.21.15.5:0/3595681659 shutdown_connections 2023-11-24T06:18:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.689+0000 7f8c9e41c640 1 -- 172.21.15.5:0/3595681659 wait complete. 2023-11-24T06:18:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.691+0000 7f8c9e41c640 1 Processor -- start 2023-11-24T06:18:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.691+0000 7f8c9e41c640 1 -- start start 2023-11-24T06:18:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.692+0000 7f8c9e41c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c840047c0 0x7f8c981f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.692+0000 7f8c9e41c640 1 --2- >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c98153330 0x7f8c981537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.692+0000 7f8c9e41c640 1 --2- >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c9815f440 0x7f8c9815f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:20.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.692+0000 7f8c9e41c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98153dd0 con 0x7f8c840047c0 2023-11-24T06:18:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.692+0000 7f8c9e41c640 1 -- --> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c981540b0 con 0x7f8c9815f440 2023-11-24T06:18:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.692+0000 7f8c9e41c640 1 -- --> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c98154390 con 0x7f8c98153330 2023-11-24T06:18:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.692+0000 7f8c97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c840047c0 0x7f8c981f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.692+0000 7f8c97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c840047c0 0x7f8c981f6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43118/0 (socket says 172.21.15.5:43118) 2023-11-24T06:18:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.692+0000 7f8c97fff640 1 -- 172.21.15.5:0/4039399755 learned_addr learned my addr 172.21.15.5:0/4039399755 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-11-24T06:18:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.693+0000 7f8c8ffff640 1 --2- 172.21.15.5:0/4039399755 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c98153330 0x7f8c981537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.693+0000 7f8c97fff640 1 -- 172.21.15.5:0/4039399755 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c98153330 msgr2=0x7f8c981537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:20.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.693+0000 7f8c97fff640 1 --2- 172.21.15.5:0/4039399755 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c98153330 0x7f8c981537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.693+0000 7f8c97fff640 1 -- 172.21.15.5:0/4039399755 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c9815f440 msgr2=0x7f8c9815f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.693+0000 7f8c97fff640 1 --2- 172.21.15.5:0/4039399755 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c9815f440 0x7f8c9815f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.693+0000 7f8c97fff640 1 -- 172.21.15.5:0/4039399755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c98154670 con 0x7f8c840047c0 2023-11-24T06:18:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.693+0000 7f8c97fff640 1 --2- 172.21.15.5:0/4039399755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c840047c0 0x7f8c981f6a60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800d570 tx=0x7f8c8800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:18:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.693+0000 7f8c957fa640 1 -- 172.21.15.5:0/4039399755 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88005480 con 0x7f8c840047c0 2023-11-24T06:18:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.694+0000 7f8c957fa640 1 -- 172.21.15.5:0/4039399755 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8c88003610 con 0x7f8c840047c0 2023-11-24T06:18:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.694+0000 7f8c957fa640 1 -- 172.21.15.5:0/4039399755 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88003980 con 0x7f8c840047c0 2023-11-24T06:18:20.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.694+0000 7f8c9e41c640 1 -- 172.21.15.5:0/4039399755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c98154950 con 0x7f8c840047c0 2023-11-24T06:18:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.694+0000 7f8c9e41c640 1 -- 172.21.15.5:0/4039399755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9815fde0 con 0x7f8c840047c0 2023-11-24T06:18:20.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.695+0000 7f8c9e41c640 1 -- 172.21.15.5:0/4039399755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c98074770 con 0x7f8c840047c0 2023-11-24T06:18:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.696+0000 7f8c957fa640 1 -- 172.21.15.5:0/4039399755 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8c88003ae0 con 0x7f8c840047c0 2023-11-24T06:18:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.697+0000 7f8c957fa640 1 --2- 172.21.15.5:0/4039399755 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c7806aed0 0x7f8c7806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T06:18:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.697+0000 7f8c957fa640 1 -- 172.21.15.5:0/4039399755 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8c880894d0 con 0x7f8c840047c0 2023-11-24T06:18:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.697+0000 7f8c8ffff640 1 --2- 172.21.15.5:0/4039399755 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c7806aed0 0x7f8c7806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T06:18:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.698+0000 7f8c8ffff640 1 --2- 172.21.15.5:0/4039399755 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c7806aed0 0x7f8c7806d390 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8c80036000 tx=0x7f8c80030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T06:18:20.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.701+0000 7f8c957fa640 1 -- 172.21.15.5:0/4039399755 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c88054680 con 0x7f8c840047c0 2023-11-24T06:18:20.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.931+0000 7f8c9e41c640 1 -- 172.21.15.5:0/4039399755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c9806e290 con 0x7f8c840047c0 2023-11-24T06:18:20.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.932+0000 7f8c957fa640 1 -- 172.21.15.5:0/4039399755 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3687 (secure 0 0 0) 0x7f8c88001230 con 0x7f8c840047c0 2023-11-24T06:18:20.935 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-11-24T06:18:20.936 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4643,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6833/2072679798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6832","nonce":2072679798},{"type":"v1","addr":"172.21.15.6:6833","nonce":2072679798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":4888,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.6:6835/3855353235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.6:6834","nonce":3855353235},{"type":"v1","addr":"172.21.15.6:6835","nonce":3855353235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6836/2394562290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2394562290},{"type":"v1","addr":"172.21.15.5:6836","nonce":2394562290}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/1134621016","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6835","nonce":1134621016},{"type":"v1","addr":"172.21.15.5:6837","nonce":1134621016}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2023-11-24T06:18:20.936 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 20 2023-11-24T06:18:20.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.935+0000 7f8c9e41c640 1 -- 172.21.15.5:0/4039399755 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c7806aed0 msgr2=0x7f8c7806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:20.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.935+0000 7f8c9e41c640 1 --2- 172.21.15.5:0/4039399755 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c7806aed0 0x7f8c7806d390 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8c80036000 tx=0x7f8c80030000 comp rx=0 tx=0).stop 2023-11-24T06:18:20.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.935+0000 7f8c9e41c640 1 -- 172.21.15.5:0/4039399755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c840047c0 msgr2=0x7f8c981f6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T06:18:20.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.935+0000 7f8c9e41c640 1 --2- 172.21.15.5:0/4039399755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c840047c0 0x7f8c981f6a60 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800d570 tx=0x7f8c8800da30 comp rx=0 tx=0).stop 2023-11-24T06:18:20.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.935+0000 7f8c9e41c640 1 -- 172.21.15.5:0/4039399755 shutdown_connections 2023-11-24T06:18:20.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.935+0000 7f8c9e41c640 1 --2- 172.21.15.5:0/4039399755 >> [v2:172.21.15.5:6832/4230007794,v1:172.21.15.5:6833/4230007794] conn(0x7f8c7806aed0 0x7f8c7806d390 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.935+0000 7f8c9e41c640 1 --2- 172.21.15.5:0/4039399755 >> [v2:172.21.15.6:3301/0,v1:172.21.15.6:6790/0] conn(0x7f8c98153330 0x7f8c981537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.935+0000 7f8c9e41c640 1 --2- 172.21.15.5:0/4039399755 >> [v2:172.21.15.6:3300/0,v1:172.21.15.6:6789/0] conn(0x7f8c9815f440 0x7f8c9815f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.935+0000 7f8c9e41c640 1 --2- 172.21.15.5:0/4039399755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c840047c0 0x7f8c981f6a60 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T06:18:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.935+0000 7f8c9e41c640 1 -- 172.21.15.5:0/4039399755 >> 172.21.15.5:0/4039399755 conn(0x7f8c980b0d80 msgr2=0x7f8c98071160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T06:18:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.935+0000 7f8c9e41c640 1 -- 172.21.15.5:0/4039399755 shutdown_connections 2023-11-24T06:18:20.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-11-24T06:18:20.935+0000 7f8c9e41c640 1 -- 172.21.15.5:0/4039399755 wait complete. 2023-11-24T06:18:20.957 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-24T06:18:20.957 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-24T06:18:20.958 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T06:18:20.958 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:20.976 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T06:18:20.977 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T06:18:20.977 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:21.047 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T06:18:21.047 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T06:18:21.047 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:21.076 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T06:18:21.077 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T06:18:21.077 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:21.147 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T06:18:21.147 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-24T06:18:21.148 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-24T06:18:21.148 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:21.200 INFO:tasks.ceph.osd.0:Stopped 2023-11-24T06:18:21.200 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-24T06:18:21.200 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:21.266 INFO:tasks.ceph.osd.1:Stopped 2023-11-24T06:18:21.267 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-24T06:18:21.267 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:21.353 INFO:tasks.ceph.osd.2:Stopped 2023-11-24T06:18:21.354 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-24T06:18:21.354 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:21.427 INFO:tasks.ceph.osd.3:Stopped 2023-11-24T06:18:21.456 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-24T06:18:21.457 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:21.534 INFO:tasks.ceph.osd.4:Stopped 2023-11-24T06:18:21.534 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-24T06:18:21.534 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:21.603 INFO:tasks.ceph.osd.5:Stopped 2023-11-24T06:18:21.603 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-24T06:18:21.603 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:21.664 INFO:tasks.ceph.osd.6:Stopped 2023-11-24T06:18:21.664 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-24T06:18:21.664 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:21.725 INFO:tasks.ceph.osd.7:Stopped 2023-11-24T06:18:21.725 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-24T06:18:21.726 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-24T06:18:21.726 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:21.796 INFO:tasks.ceph.mgr.y:Stopped 2023-11-24T06:18:21.797 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-24T06:18:21.797 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:25.094 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2023-11-24T06:18:27.898 INFO:tasks.ceph.mgr.x:Stopped 2023-11-24T06:18:27.898 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-24T06:18:27.899 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-24T06:18:27.899 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:27.948 INFO:tasks.ceph.mon.a:Stopped 2023-11-24T06:18:27.949 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-24T06:18:27.949 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:27.982 INFO:tasks.ceph.mon.b:Stopped 2023-11-24T06:18:27.982 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-24T06:18:27.982 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T06:18:28.056 INFO:tasks.ceph.mon.c:Stopped 2023-11-24T06:18:28.056 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-24T06:18:28.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-11-24T06:18:28.177 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T06:18:28.177 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-24T06:18:28.445 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T06:18:28.445 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-24T06:18:28.615 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T06:18:28.615 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-24T06:18:28.769 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T06:18:28.770 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-24T06:18:28.992 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T06:18:28.992 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-24T06:18:29.258 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T06:18:29.258 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-24T06:18:29.417 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T06:18:29.417 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-24T06:18:29.590 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T06:18:29.590 DEBUG:teuthology.orchestra.run.smithi006:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-24T06:18:29.747 INFO:tasks.ceph:Archiving mon data... 2023-11-24T06:18:29.748 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552/data/mon.a.tgz 2023-11-24T06:18:29.749 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2023-11-24T06:18:29.813 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.RnaROihhYx 2023-11-24T06:18:29.813 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.RnaROihhYx 2023-11-24T06:18:30.022 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.RnaROihhYx 2023-11-24T06:18:30.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2023-11-24T06:18:30.188 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.RnaROihhYx is 330KB 2023-11-24T06:18:30.301 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.RnaROihhYx 2023-11-24T06:18:30.363 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552/data/mon.b.tgz 2023-11-24T06:18:30.363 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2023-11-24T06:18:30.425 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.1uXGfyTGnw 2023-11-24T06:18:30.426 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.1uXGfyTGnw 2023-11-24T06:18:30.641 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.1uXGfyTGnw 2023-11-24T06:18:30.802 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.1uXGfyTGnw is 442KB 2023-11-24T06:18:30.916 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.1uXGfyTGnw 2023-11-24T06:18:30.980 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552/data/mon.c.tgz 2023-11-24T06:18:30.980 DEBUG:teuthology.orchestra.run.smithi006:> mktemp 2023-11-24T06:18:31.036 INFO:teuthology.orchestra.run.smithi006.stdout:/tmp/tmp.dyJzMrBuJH 2023-11-24T06:18:31.037 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.dyJzMrBuJH 2023-11-24T06:18:31.238 DEBUG:teuthology.orchestra.run.smithi006:> sudo chmod 0666 /tmp/tmp.dyJzMrBuJH 2023-11-24T06:18:31.406 DEBUG:teuthology.orchestra.remote:smithi006:/tmp/tmp.dyJzMrBuJH is 442KB 2023-11-24T06:18:31.521 DEBUG:teuthology.orchestra.run.smithi006:> rm -fr /tmp/tmp.dyJzMrBuJH 2023-11-24T06:18:31.583 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-24T06:18:31.583 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T06:18:31.592 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T06:18:31.694 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-24T06:18:31.694 DEBUG:teuthology.orchestra.run.smithi005:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T06:18:31.700 DEBUG:teuthology.orchestra.run.smithi006:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T06:18:31.798 INFO:teuthology.orchestra.run.smithi005.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T06:18:31.815 INFO:teuthology.orchestra.run.smithi006.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T06:18:31.827 INFO:tasks.ceph:Archiving crash dumps... 2023-11-24T06:18:31.829 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552/remote/smithi005/crash 2023-11-24T06:18:31.830 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T06:18:31.920 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552/remote/smithi006/crash 2023-11-24T06:18:31.921 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T06:18:32.026 INFO:tasks.ceph:Shutting down logrotate 2023-11-24T06:18:32.027 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:18:32.036 DEBUG:teuthology.orchestra.run.smithi006:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T06:18:32.182 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T06:18:32.291 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T06:18:32.406 INFO:tasks.ceph:Compressing logs... 2023-11-24T06:18:32.406 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T06:18:32.416 DEBUG:teuthology.orchestra.run.smithi006:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T06:18:32.510 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94060.log 2023-11-24T06:18:32.511 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98914.log 2023-11-24T06:18:32.512 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94060.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94060.log.gz 2023-11-24T06:18:32.513 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96937.log 2023-11-24T06:18:32.514 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97948.log 2023-11-24T06:18:32.514 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98914.log.gz 2023-11-24T06:18:32.515 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.96937.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96937.log.gz 2023-11-24T06:18:32.515 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97399.log 2023-11-24T06:18:32.516 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-24T06:18:32.517 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94402.log 2023-11-24T06:18:32.517 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97948.log.gz 2023-11-24T06:18:32.518 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.94670.log 2023-11-24T06:18:32.518 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100866.log 2023-11-24T06:18:32.519 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97399.log.gz 2023-11-24T06:18:32.519 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88792.log 2023-11-24T06:18:32.520 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94402.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94402.log.gz 2023-11-24T06:18:32.520 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99832.log 2023-11-24T06:18:32.522 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100602.log 2023-11-24T06:18:32.522 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100866.log.gz 2023-11-24T06:18:32.522 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.1.94670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.93918.log 2023-11-24T06:18:32.522 INFO:teuthology.orchestra.run.smithi006.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.1.94670.log.gz 2023-11-24T06:18:32.523 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.88792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88872.log 2023-11-24T06:18:32.523 INFO:teuthology.orchestra.run.smithi006.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.88792.log.gz 2023-11-24T06:18:32.524 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.3.93918.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T06:18:32.524 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99832.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.99832.log.gz 2023-11-24T06:18:32.525 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99706.log 2023-11-24T06:18:32.525 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100602.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-24T06:18:32.525 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100602.log.gz 2023-11-24T06:18:32.526 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92957.log 2023-11-24T06:18:32.526 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99706.log.gz 2023-11-24T06:18:32.527 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.88872.log: 91.9% -- replaced with /var/log/ceph/ceph-client.3.93918.log.gz 2023-11-24T06:18:32.527 INFO:teuthology.orchestra.run.smithi006.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88872.log.gz 2023-11-24T06:18:32.527 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-24T06:18:32.527 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88912.log 2023-11-24T06:18:32.529 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-24T06:18:32.529 INFO:teuthology.orchestra.run.smithi006.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T06:18:32.530 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.88912.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88912.log.gz 2023-11-24T06:18:32.530 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.93466.log 2023-11-24T06:18:32.531 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92611.log 2023-11-24T06:18:32.531 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.92957.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92957.log.gz 2023-11-24T06:18:32.531 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100056.log 2023-11-24T06:18:32.531 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.92611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97231.log 2023-11-24T06:18:32.531 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.92611.log.gz 2023-11-24T06:18:32.532 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100056.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100056.log.gz 2023-11-24T06:18:32.532 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98562.log 2023-11-24T06:18:32.534 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94047.log 2023-11-24T06:18:32.534 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97231.log.gz 2023-11-24T06:18:32.534 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98562.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98562.log.gz 2023-11-24T06:18:32.534 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97483.log 2023-11-24T06:18:32.535 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88712.log 2023-11-24T06:18:32.535 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.1.93466.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-24T06:18:32.535 INFO:teuthology.orchestra.run.smithi006.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.1.93466.log.gz 2023-11-24T06:18:32.536 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.88712.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-24T06:18:32.536 INFO:teuthology.orchestra.run.smithi006.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88712.log.gz 2023-11-24T06:18:32.537 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88752.log 2023-11-24T06:18:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94047.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94047.log.gz 2023-11-24T06:18:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93711.log 2023-11-24T06:18:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97483.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97483.log.gz 2023-11-24T06:18:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102071.log 2023-11-24T06:18:32.539 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93711.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93711.log.gz 2023-11-24T06:18:32.539 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96727.log 2023-11-24T06:18:32.540 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102071.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102071.log.gz 2023-11-24T06:18:32.540 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92788.log 2023-11-24T06:18:32.541 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-24T06:18:32.541 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.88752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88672.log 2023-11-24T06:18:32.541 INFO:teuthology.orchestra.run.smithi006.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.88752.log.gz 2023-11-24T06:18:32.542 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.96727.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96727.log.gz 2023-11-24T06:18:32.542 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99118.log 2023-11-24T06:18:32.543 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T06:18:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.92788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99034.log 2023-11-24T06:18:32.543 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92788.log.gz 2023-11-24T06:18:32.544 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.88672.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88672.log.gz 2023-11-24T06:18:32.545 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102293.log 2023-11-24T06:18:32.545 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99118.log.gz 2023-11-24T06:18:32.545 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93043.log 2023-11-24T06:18:32.546 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99034.log.gz 2023-11-24T06:18:32.547 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96643.log 2023-11-24T06:18:32.547 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102293.log.gz 2023-11-24T06:18:32.547 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph.log: 92.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-24T06:18:32.547 INFO:teuthology.orchestra.run.smithi006.stderr: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T06:18:32.548 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88952.log 2023-11-24T06:18:32.549 INFO:teuthology.orchestra.run.smithi006.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88832.log 2023-11-24T06:18:32.549 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93043.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93043.log.gz 2023-11-24T06:18:32.549 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102149.log 2023-11-24T06:18:32.550 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.88952.log: gzip -5 --verbose 87.9% -- /var/log/ceph/ceph-mon.b.log -- replaced with /var/log/ceph/ceph-client.admin.88952.log.gz 2023-11-24T06:18:32.550 INFO:teuthology.orchestra.run.smithi006.stderr: 2023-11-24T06:18:32.551 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.96643.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96643.log.gz 2023-11-24T06:18:32.551 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100136.log 2023-11-24T06:18:32.552 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94326.log 2023-11-24T06:18:32.552 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102149.log.gz 2023-11-24T06:18:32.553 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-client.admin.88832.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-24T06:18:32.553 INFO:teuthology.orchestra.run.smithi006.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.88832.log.gz 2023-11-24T06:18:32.553 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100136.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.100136.log.gz 2023-11-24T06:18:32.553 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99328.log 2023-11-24T06:18:32.554 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-24T06:18:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99664.log 2023-11-24T06:18:32.555 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94326.log.gz 2023-11-24T06:18:32.555 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101726.log 2023-11-24T06:18:32.556 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99328.log.gz 2023-11-24T06:18:32.556 INFO:teuthology.orchestra.run.smithi006.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 90.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-24T06:18:32.557 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99664.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99664.log.gz 2023-11-24T06:18:32.557 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93731.log 2023-11-24T06:18:32.558 INFO:teuthology.orchestra.run.smithi006.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-24T06:18:32.559 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.101726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103717.log 2023-11-24T06:18:32.559 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101726.log.gz 2023-11-24T06:18:32.559 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93731.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93731.log.gz -5 2023-11-24T06:18:32.560 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.102251.log 2023-11-24T06:18:32.561 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103717.log: gzip -5 87.9% --verbose -- replaced with /var/log/ceph/ceph-client.admin.103717.log.gz -- 2023-11-24T06:18:32.561 INFO:teuthology.orchestra.run.smithi005.stderr: /var/log/ceph/ceph-client.admin.100222.log 2023-11-24T06:18:32.562 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101184.log 2023-11-24T06:18:32.562 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102251.log.gz 2023-11-24T06:18:32.563 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100222.log: gzip -5 --verbose 88.5% -- -- replaced with /var/log/ceph/ceph-client.admin.100222.log.gz /var/log/ceph/ceph-client.admin.104053.log 2023-11-24T06:18:32.564 INFO:teuthology.orchestra.run.smithi005.stderr: 2023-11-24T06:18:32.565 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.101184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97764.log 2023-11-24T06:18:32.565 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101184.log.gz 2023-11-24T06:18:32.566 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.104053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93730.log 2023-11-24T06:18:32.566 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104053.log.gz 2023-11-24T06:18:32.567 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103801.log 2023-11-24T06:18:32.567 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97764.log.gz 2023-11-24T06:18:32.569 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93730.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93730.log.gz -5 2023-11-24T06:18:32.569 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.102788.log 2023-11-24T06:18:32.570 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98830.log 2023-11-24T06:18:32.570 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103801.log.gz 2023-11-24T06:18:32.571 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104125.log 2023-11-24T06:18:32.571 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102788.log.gz 2023-11-24T06:18:32.572 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97722.log 2023-11-24T06:18:32.573 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98830.log.gz 2023-11-24T06:18:32.574 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.104125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100178.log 2023-11-24T06:18:32.574 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104125.log.gz 2023-11-24T06:18:32.575 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95163.log 2023-11-24T06:18:32.575 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97722.log.gz 2023-11-24T06:18:32.576 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97680.log 2023-11-24T06:18:32.577 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100178.log.gz 2023-11-24T06:18:32.578 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.95163.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95163.log.gz 2023-11-24T06:18:32.578 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104165.log 2023-11-24T06:18:32.579 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97680.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.82542.log 2023-11-24T06:18:32.579 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97680.log.gz 2023-11-24T06:18:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.104165.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104165.log.gz 2023-11-24T06:18:32.580 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98604.log 2023-11-24T06:18:32.581 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.tmp-client.admin.82542.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.82542.log.gz 2023-11-24T06:18:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96491.log 2023-11-24T06:18:32.583 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94893.log 2023-11-24T06:18:32.583 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98604.log.gz 2023-11-24T06:18:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.96491.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96491.log.gz 2023-11-24T06:18:32.584 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94641.log 2023-11-24T06:18:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94893.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94893.log.gz 2023-11-24T06:18:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97021.log 2023-11-24T06:18:32.587 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102830.log 2023-11-24T06:18:32.587 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94641.log.gz 2023-11-24T06:18:32.588 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93213.log 2023-11-24T06:18:32.589 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97021.log.gz 2023-11-24T06:18:32.590 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93727.log 2023-11-24T06:18:32.590 INFO:teuthology.orchestra.run.smithi005.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.102830.log.gz 2023-11-24T06:18:32.591 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93213.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93213.log.gz 2023-11-24T06:18:32.591 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101076.log 2023-11-24T06:18:32.592 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103759.log 2023-11-24T06:18:32.592 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93727.log.gz 2023-11-24T06:18:32.594 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.101076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98362.log 2023-11-24T06:18:32.594 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101076.log.gz 2023-11-24T06:18:32.595 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103759.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103759.log.gz 2023-11-24T06:18:32.595 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98404.log 2023-11-24T06:18:32.596 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88873.log 2023-11-24T06:18:32.596 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98362.log.gz 2023-11-24T06:18:32.597 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98404.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98404.log.gz 2023-11-24T06:18:32.597 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100390.log 2023-11-24T06:18:32.598 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.88873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98068.log 2023-11-24T06:18:32.598 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88873.log.gz 2023-11-24T06:18:32.599 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99202.log 2023-11-24T06:18:32.600 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100390.log.gz 2023-11-24T06:18:32.601 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99580.log 2023-11-24T06:18:32.601 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98068.log.gz 2023-11-24T06:18:32.602 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99202.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99202.log.gz 2023-11-24T06:18:32.602 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98446.log 2023-11-24T06:18:32.603 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103297.log 2023-11-24T06:18:32.604 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99580.log.gz 2023-11-24T06:18:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94809.log 2023-11-24T06:18:32.605 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98446.log.gz 2023-11-24T06:18:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103297.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103297.log.gz 2023-11-24T06:18:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103255.log 2023-11-24T06:18:32.607 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97315.log 2023-11-24T06:18:32.607 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94809.log.gz 2023-11-24T06:18:32.608 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103255.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103255.log.gz 2023-11-24T06:18:32.608 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.98110.log 2023-11-24T06:18:32.610 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94066.log 2023-11-24T06:18:32.610 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97315.log.gz 2023-11-24T06:18:32.611 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98110.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98110.log.gz 2023-11-24T06:18:32.611 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96449.log 2023-11-24T06:18:32.612 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102536.log 2023-11-24T06:18:32.612 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94066.log.gz 2023-11-24T06:18:32.613 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.96449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93171.log 2023-11-24T06:18:32.613 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96449.log.gz 2023-11-24T06:18:32.615 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100992.log 2023-11-24T06:18:32.615 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102536.log.gz 2023-11-24T06:18:32.618 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98152.log 2023-11-24T06:18:32.618 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93171.log.gz 2023-11-24T06:18:32.618 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100992.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100992.log.gz 2023-11-24T06:18:32.618 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92873.log 2023-11-24T06:18:32.619 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98152.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98152.log.gz 2023-11-24T06:18:32.619 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93423.log 2023-11-24T06:18:32.620 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.92873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102890.log 2023-11-24T06:18:32.621 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92873.log.gz 2023-11-24T06:18:32.622 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93423.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93423.log.gz 2023-11-24T06:18:32.622 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103059.log 2023-11-24T06:18:32.623 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103549.log 2023-11-24T06:18:32.623 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.102890.log.gz 2023-11-24T06:18:32.624 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103059.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T06:18:32.624 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103059.log.gz 2023-11-24T06:18:32.625 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94053.log 2023-11-24T06:18:32.626 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103549.log.gz 2023-11-24T06:18:32.626 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94403.log 2023-11-24T06:18:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94053.log: 88.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T06:18:32.628 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94599.log 2023-11-24T06:18:32.628 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94053.log.gz 2023-11-24T06:18:32.629 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94381.log 2023-11-24T06:18:32.630 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94403.log.gz 2023-11-24T06:18:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94599.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94599.log.gz 2023-11-24T06:18:32.631 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100474.log 2023-11-24T06:18:32.632 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100516.log 2023-11-24T06:18:32.632 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94381.log.gz 2023-11-24T06:18:32.633 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99790.log 2023-11-24T06:18:32.633 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100474.log.gz 2023-11-24T06:18:32.634 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99076.log 2023-11-24T06:18:32.635 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100516.log.gz 2023-11-24T06:18:32.635 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94373.log 2023-11-24T06:18:32.636 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99790.log.gz 2023-11-24T06:18:32.637 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98194.log 2023-11-24T06:18:32.637 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99076.log.gz 2023-11-24T06:18:32.639 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99370.log 2023-11-24T06:18:32.639 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94373.log.gz 2023-11-24T06:18:32.640 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98194.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98194.log.gz 2023-11-24T06:18:32.640 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100646.log 2023-11-24T06:18:32.641 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99370.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99370.log.gz 2023-11-24T06:18:32.641 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99622.log 2023-11-24T06:18:32.642 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95121.log 2023-11-24T06:18:32.643 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100646.log.gz 2023-11-24T06:18:32.644 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99622.log: gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.103885.log 2023-11-24T06:18:32.644 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99622.log.gz 2023-11-24T06:18:32.645 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.95121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96895.log 2023-11-24T06:18:32.645 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95121.log.gz 2023-11-24T06:18:32.646 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104207.log 2023-11-24T06:18:32.646 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103885.log.gz 2023-11-24T06:18:32.647 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.96895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102933.log 2023-11-24T06:18:32.648 INFO:teuthology.orchestra.run.smithi006.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-24T06:18:32.648 INFO:teuthology.orchestra.run.smithi005.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96895.log.gz 2023-11-24T06:18:32.649 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.104207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102029.log 2023-11-24T06:18:32.649 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104207.log.gz 2023-11-24T06:18:32.650 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93297.log 2023-11-24T06:18:32.650 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.102933.log.gz 2023-11-24T06:18:32.651 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102029.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102029.log.gz 2023-11-24T06:18:32.651 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102578.log 2023-11-24T06:18:32.652 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99244.log 2023-11-24T06:18:32.653 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93297.log.gz 2023-11-24T06:18:32.654 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97906.log 2023-11-24T06:18:32.654 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102578.log.gz 2023-11-24T06:18:32.655 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99244.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.99244.log.gz -5 2023-11-24T06:18:32.655 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.98992.log 2023-11-24T06:18:32.656 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99454.log 2023-11-24T06:18:32.656 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97906.log.gz 2023-11-24T06:18:32.657 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99538.log 2023-11-24T06:18:32.658 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98992.log.gz 2023-11-24T06:18:32.658 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100950.log 2023-11-24T06:18:32.659 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99454.log.gz 2023-11-24T06:18:32.660 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102335.log 2023-11-24T06:18:32.660 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99538.log.gz 2023-11-24T06:18:32.661 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99160.log 2023-11-24T06:18:32.661 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100950.log.gz 2023-11-24T06:18:32.662 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102335.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.102335.log.gz -5 2023-11-24T06:18:32.662 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.97063.log 2023-11-24T06:18:32.663 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96853.log 2023-11-24T06:18:32.663 INFO:teuthology.orchestra.run.smithi005.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99160.log.gz 2023-11-24T06:18:32.665 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93716.log 2023-11-24T06:18:32.665 INFO:teuthology.orchestra.run.smithi005.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97063.log.gz 2023-11-24T06:18:32.666 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.96853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94391.log 2023-11-24T06:18:32.666 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96853.log.gz 2023-11-24T06:18:32.667 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93716.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93716.log.gz 2023-11-24T06:18:32.668 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92830.log 2023-11-24T06:18:32.668 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94391.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-24T06:18:32.669 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94391.log.gz 2023-11-24T06:18:32.670 INFO:teuthology.orchestra.run.smithi006.stderr: 93.5% 93.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-24T06:18:32.670 INFO:teuthology.orchestra.run.smithi006.stderr: 2023-11-24T06:18:32.670 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.92830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97357.log 2023-11-24T06:18:32.671 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92830.log.gz 2023-11-24T06:18:32.671 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94400.log 2023-11-24T06:18:32.673 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97357.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97357.log.gz 2023-11-24T06:18:32.673 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93549.log 2023-11-24T06:18:32.674 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98872.log 2023-11-24T06:18:32.675 INFO:teuthology.orchestra.run.smithi005.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94400.log.gz 2023-11-24T06:18:32.675 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93549.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93549.log.gz 2023-11-24T06:18:32.676 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97189.log 2023-11-24T06:18:32.677 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98872.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98872.log.gz 2023-11-24T06:18:32.677 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95939.log 2023-11-24T06:18:32.678 INFO:teuthology.orchestra.run.smithi006.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-24T06:18:32.679 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97189.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97189.log.gz 2023-11-24T06:18:32.679 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93129.log 2023-11-24T06:18:32.680 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.95939.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95939.log.gz 2023-11-24T06:18:32.680 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100824.log 2023-11-24T06:18:32.681 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93129.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93129.log.gz 2023-11-24T06:18:32.682 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99412.log 2023-11-24T06:18:32.683 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100824.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100824.log.gz 2023-11-24T06:18:32.683 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99286.log 2023-11-24T06:18:32.684 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99412.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99412.log.gz 2023-11-24T06:18:32.685 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103213.log 2023-11-24T06:18:32.686 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99286.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99286.log.gz 2023-11-24T06:18:32.686 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98236.log 2023-11-24T06:18:32.687 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103213.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103213.log.gz 2023-11-24T06:18:32.688 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103507.log 2023-11-24T06:18:32.689 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98236.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98236.log.gz 2023-11-24T06:18:32.689 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102975.log 2023-11-24T06:18:32.690 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103507.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103507.log.gz 2023-11-24T06:18:32.690 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97441.log 2023-11-24T06:18:32.692 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101768.log 2023-11-24T06:18:32.692 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102975.log.gz 2023-11-24T06:18:32.693 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97441.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97441.log.gz 2023-11-24T06:18:32.693 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99748.log 2023-11-24T06:18:32.695 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.101768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100348.log 2023-11-24T06:18:32.695 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101768.log.gz 2023-11-24T06:18:32.696 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99748.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99748.log.gz 2023-11-24T06:18:32.696 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102620.log 2023-11-24T06:18:32.697 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100432.log 2023-11-24T06:18:32.698 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100348.log.gz 2023-11-24T06:18:32.698 INFO:teuthology.orchestra.run.smithi006.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-24T06:18:32.699 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102620.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102620.log.gz 2023-11-24T06:18:32.699 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98278.log 2023-11-24T06:18:32.700 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103339.log 2023-11-24T06:18:32.701 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100432.log.gz 2023-11-24T06:18:32.701 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98278.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98278.log.gz 2023-11-24T06:18:32.702 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95061.log 2023-11-24T06:18:32.703 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98788.log 2023-11-24T06:18:32.703 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103339.log.gz 2023-11-24T06:18:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.95061.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95061.log.gz 2023-11-24T06:18:32.704 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103675.log 2023-11-24T06:18:32.706 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102410.log 2023-11-24T06:18:32.706 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98788.log.gz 2023-11-24T06:18:32.707 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103675.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103675.log.gz 2023-11-24T06:18:32.707 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93465.log 2023-11-24T06:18:32.708 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100558.log 2023-11-24T06:18:32.709 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102410.log.gz 2023-11-24T06:18:32.710 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93465.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93465.log.gzgzip 2023-11-24T06:18:32.710 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-24T06:18:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93724.log 2023-11-24T06:18:32.711 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100558.log.gz 2023-11-24T06:18:32.712 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103927.log 2023-11-24T06:18:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100782.log 2023-11-24T06:18:32.714 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93724.log.gz 2023-11-24T06:18:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103927.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103927.log.gz 2023-11-24T06:18:32.716 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101987.log 2023-11-24T06:18:32.717 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102191.log 2023-11-24T06:18:32.718 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100782.log.gz 2023-11-24T06:18:32.719 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.101987.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101987.log.gz 2023-11-24T06:18:32.720 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96979.log 2023-11-24T06:18:32.721 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102191.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102191.log.gz 2023-11-24T06:18:32.721 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99962.log 2023-11-24T06:18:32.723 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.96979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97990.log 2023-11-24T06:18:32.723 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96979.log.gz 2023-11-24T06:18:32.725 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99962.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99962.log.gz 2023-11-24T06:18:32.725 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94067.log 2023-11-24T06:18:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99916.log 2023-11-24T06:18:32.727 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97990.log.gz 2023-11-24T06:18:32.728 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104011.log 2023-11-24T06:18:32.728 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94067.log.gz 2023-11-24T06:18:32.729 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93255.log 2023-11-24T06:18:32.730 INFO:teuthology.orchestra.run.smithi005.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99916.log.gz 2023-11-24T06:18:32.731 INFO:teuthology.orchestra.run.smithi006.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-24T06:18:32.732 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.104011.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104011.log.gzgzip 2023-11-24T06:18:32.732 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.96811.log 2023-11-24T06:18:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94977.log 89.0% 2023-11-24T06:18:32.733 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93255.log.gz 2023-11-24T06:18:32.735 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.96811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96769.log 2023-11-24T06:18:32.735 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96811.log.gz 2023-11-24T06:18:32.736 INFO:teuthology.orchestra.run.smithi006.stderr: 2023-11-24T06:18:32.736 INFO:teuthology.orchestra.run.smithi006.stderr:real 0m0.262s 2023-11-24T06:18:32.736 INFO:teuthology.orchestra.run.smithi006.stderr:user 0m0.862s 2023-11-24T06:18:32.736 INFO:teuthology.orchestra.run.smithi006.stderr:sys 0m0.211s 2023-11-24T06:18:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94977.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94977.log.gz 2023-11-24T06:18:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96601.log 2023-11-24T06:18:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.96769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102704.log 2023-11-24T06:18:32.739 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96769.log.gz 2023-11-24T06:18:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.96601.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96601.log.gz 2023-11-24T06:18:32.740 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94851.log 2023-11-24T06:18:32.741 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94683.log 2023-11-24T06:18:32.742 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102704.log.gz 2023-11-24T06:18:32.743 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102452.log 2023-11-24T06:18:32.743 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94851.log.gz 2023-11-24T06:18:32.745 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94683.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.101118.log -- replaced with /var/log/ceph/ceph-client.admin.94683.log.gz 2023-11-24T06:18:32.745 INFO:teuthology.orchestra.run.smithi005.stderr: 2023-11-24T06:18:32.747 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102452.log: gzip 88.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.102452.log.gz -- 2023-11-24T06:18:32.747 INFO:teuthology.orchestra.run.smithi005.stderr: /var/log/ceph/ceph-client.admin.93720.log 2023-11-24T06:18:32.748 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.101118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102662.log 2023-11-24T06:18:32.749 INFO:teuthology.orchestra.run.smithi005.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.101118.log.gz 2023-11-24T06:18:32.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93720.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-24T06:18:32.750 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93720.log.gz 2023-11-24T06:18:32.751 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101903.log 2023-11-24T06:18:32.752 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102662.log.gz 2023-11-24T06:18:32.753 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94064.log 2023-11-24T06:18:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.101903.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101903.log.gz 2023-11-24T06:18:32.755 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98320.log 2023-11-24T06:18:32.757 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94064.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94064.log.gz 2023-11-24T06:18:32.757 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95205.log 2023-11-24T06:18:32.757 INFO:teuthology.orchestra.run.smithi005.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-24T06:18:32.758 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98320.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98320.log.gz 2023-11-24T06:18:32.759 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100306.log 2023-11-24T06:18:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.95205.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95205.log.gz 2023-11-24T06:18:32.760 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99874.log 2023-11-24T06:18:32.761 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100306.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100306.log.gz 2023-11-24T06:18:32.762 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92915.log 2023-11-24T06:18:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99874.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99874.log.gz 2023-11-24T06:18:32.763 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97578.log 2023-11-24T06:18:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.92915.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.92915.log.gz 2023-11-24T06:18:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102746.log 2023-11-24T06:18:32.766 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97578.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97578.log.gz 2023-11-24T06:18:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.95799.log 2023-11-24T06:18:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102746.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102746.log.gz 2023-11-24T06:18:32.769 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100696.log 2023-11-24T06:18:32.770 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.95799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98520.log 2023-11-24T06:18:32.770 INFO:teuthology.orchestra.run.smithi005.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.95799.log.gz 2023-11-24T06:18:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94062.log 2023-11-24T06:18:32.771 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100696.log.gz 2023-11-24T06:18:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98520.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98520.log.gz 2023-11-24T06:18:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93087.log 2023-11-24T06:18:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94062.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94062.log.gz 2023-11-24T06:18:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101945.log 2023-11-24T06:18:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101642.log 2023-11-24T06:18:32.776 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93087.log.gz 2023-11-24T06:18:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.101945.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101945.log.gz 2023-11-24T06:18:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96685.log 2023-11-24T06:18:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.101642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97147.log 2023-11-24T06:18:32.778 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101642.log.gz 2023-11-24T06:18:32.779 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.96685.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96685.log.gz 2023-11-24T06:18:32.779 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94725.log 2023-11-24T06:18:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97105.log 2023-11-24T06:18:32.781 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97147.log.gz 2023-11-24T06:18:32.782 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94725.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94725.log.gz 2023-11-24T06:18:32.782 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89000.log 2023-11-24T06:18:32.783 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99496.log 2023-11-24T06:18:32.784 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.97105.log.gz 2023-11-24T06:18:32.784 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.89000.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89000.log.gz 2023-11-24T06:18:32.785 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103633.log 2023-11-24T06:18:32.786 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.99496.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-24T06:18:32.786 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99496.log.gz 2023-11-24T06:18:32.787 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103633.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103633.log.gz 2023-11-24T06:18:32.787 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94767.log 2023-11-24T06:18:32.788 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100264.log 2023-11-24T06:18:32.789 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94767.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94767.log.gz 2023-11-24T06:18:32.790 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97822.log 2023-11-24T06:18:32.791 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101684.log 2023-11-24T06:18:32.791 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100264.log.gz 2023-11-24T06:18:32.792 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97822.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.97822.log.gz 2023-11-24T06:18:32.792 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97864.log 2023-11-24T06:18:32.793 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.101684.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T06:18:32.794 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101684.log.gz 2023-11-24T06:18:32.794 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97864.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97864.log.gz 2023-11-24T06:18:32.795 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92999.log 2023-11-24T06:18:32.796 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103423.log 2023-11-24T06:18:32.797 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.92999.log: 85.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T06:18:32.798 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92999.log.gz 2023-11-24T06:18:32.798 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92707.log 2023-11-24T06:18:32.799 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.96345.log 2023-11-24T06:18:32.799 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103423.log.gz 2023-11-24T06:18:32.801 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.92707.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-24T06:18:32.801 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92707.log.gz 2023-11-24T06:18:32.802 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.2.96345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97638.log 2023-11-24T06:18:32.803 INFO:teuthology.orchestra.run.smithi005.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-24T06:18:32.803 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.2.log: 92.3% -- replaced with /var/log/ceph/ceph-client.2.96345.log.gz 2023-11-24T06:18:32.804 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-24T06:18:32.805 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97638.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97638.log.gz 2023-11-24T06:18:32.805 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93507.log 2023-11-24T06:18:32.807 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101824.log 2023-11-24T06:18:32.807 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-24T06:18:32.808 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93507.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93507.log.gz 2023-11-24T06:18:32.808 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94388.log 2023-11-24T06:18:32.810 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.101824.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101824.log.gz 2023-11-24T06:18:32.810 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103969.log 2023-11-24T06:18:32.811 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94388.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94388.log.gz 2023-11-24T06:18:32.812 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97273.log 2023-11-24T06:18:32.813 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103969.log: 91.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-24T06:18:32.813 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103969.log.gz 2023-11-24T06:18:32.813 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98646.log 2023-11-24T06:18:32.815 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.97273.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97273.log.gz 2023-11-24T06:18:32.815 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101034.log 2023-11-24T06:18:32.816 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98646.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98646.log.gz 2023-11-24T06:18:32.816 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93381.log 2023-11-24T06:18:32.817 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.101034.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101034.log.gz 2023-11-24T06:18:32.819 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98746.log 2023-11-24T06:18:32.821 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93381.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93381.log.gz -5 2023-11-24T06:18:32.821 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.94935.log 2023-11-24T06:18:32.822 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98746.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98746.log.gz 2023-11-24T06:18:32.822 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102494.log 2023-11-24T06:18:32.823 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95247.log 2023-11-24T06:18:32.824 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94935.log.gz 2023-11-24T06:18:32.825 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.102494.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102494.log.gz 2023-11-24T06:18:32.825 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92519.log 2023-11-24T06:18:32.826 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.95247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101600.log 2023-11-24T06:18:32.827 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.95247.log.gz 2023-11-24T06:18:32.828 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.92519.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92519.log.gz 2023-11-24T06:18:32.828 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103017.log 2023-11-24T06:18:32.829 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.101600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100908.log 2023-11-24T06:18:32.829 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101600.log.gz 2023-11-24T06:18:32.830 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103017.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103017.log.gz 2023-11-24T06:18:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94050.log 2023-11-24T06:18:32.831 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100908.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100908.log.gz 2023-11-24T06:18:32.832 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93339.log 2023-11-24T06:18:32.833 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.94050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93728.log 2023-11-24T06:18:32.834 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94050.log.gz 2023-11-24T06:18:32.834 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93339.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93339.log.gz 2023-11-24T06:18:32.835 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103381.log 2023-11-24T06:18:32.836 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.93728.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93728.log.gzgzip 2023-11-24T06:18:32.837 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.95019.log 2023-11-24T06:18:32.838 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103381.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103381.log.gz 2023-11-24T06:18:32.838 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88616.log 2023-11-24T06:18:32.839 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.95019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95350.log 2023-11-24T06:18:32.840 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95019.log.gz 2023-11-24T06:18:32.841 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.88616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103465.log 2023-11-24T06:18:32.841 INFO:teuthology.orchestra.run.smithi005.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.88616.log.gz 2023-11-24T06:18:32.842 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.95350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100738.log 2023-11-24T06:18:32.842 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95350.log.gz 2023-11-24T06:18:32.843 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103465.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103465.log.gz 2023-11-24T06:18:32.844 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-24T06:18:32.845 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.100738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.101461.log 2023-11-24T06:18:32.845 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100738.log.gz 2023-11-24T06:18:32.847 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98688.log 2023-11-24T06:18:32.848 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.101461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103591.log 2023-11-24T06:18:32.849 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.0.101461.log.gz 2023-11-24T06:18:32.850 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.98688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103843.log 2023-11-24T06:18:32.850 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98688.log.gz 2023-11-24T06:18:32.851 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103591.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103591.log.gz 2023-11-24T06:18:32.853 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.103843.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103843.log.gz 2023-11-24T06:18:32.865 INFO:teuthology.orchestra.run.smithi005.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-24T06:18:32.883 INFO:teuthology.orchestra.run.smithi005.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-24T06:18:32.894 INFO:teuthology.orchestra.run.smithi005.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-24T06:18:32.914 INFO:teuthology.orchestra.run.smithi005.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-24T06:18:32.918 INFO:teuthology.orchestra.run.smithi005.stderr: 2023-11-24T06:18:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:real 0m0.453s 2023-11-24T06:18:32.918 INFO:teuthology.orchestra.run.smithi005.stderr:user 0m0.864s 2023-11-24T06:18:32.919 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m0.827s 2023-11-24T06:18:32.920 INFO:tasks.ceph:Archiving logs... 2023-11-24T06:18:32.920 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552/remote/smithi005/log 2023-11-24T06:18:32.921 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T06:18:33.290 DEBUG:teuthology.misc:Transferring archived files from smithi006:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552/remote/smithi006/log 2023-11-24T06:18:33.291 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T06:18:33.505 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-24T06:18:33.526 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-24T06:18:33.527 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T06:18:33.536 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T06:18:33.688 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-24T06:18:33.698 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-24T06:18:33.698 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:18:33.709 DEBUG:teuthology.orchestra.run.smithi006:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T06:18:33.761 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:18:33.769 INFO:teuthology.orchestra.run.smithi006.stderr:bash: line 1: ntpq: command not found 2023-11-24T06:18:33.771 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:18:33.771 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2023-11-24T06:18:33.771 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 24 +2103us[+2103us] +/- 74ms 2023-11-24T06:18:33.771 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -1714us[-1721us] +/- 62ms 2023-11-24T06:18:33.771 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 -100us[ -100us] +/- 94ms 2023-11-24T06:18:33.771 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:18:33.779 INFO:teuthology.orchestra.run.smithi006.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T06:18:33.780 INFO:teuthology.orchestra.run.smithi006.stdout:=============================================================================== 2023-11-24T06:18:33.780 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 24 +2131us[+2136us] +/- 74ms 2023-11-24T06:18:33.780 INFO:teuthology.orchestra.run.smithi006.stdout:^* hv02.front.sepia.ceph.com 3 6 377 24 -1686us[-1681us] +/- 62ms 2023-11-24T06:18:33.780 INFO:teuthology.orchestra.run.smithi006.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 -105us[ -100us] +/- 94ms 2023-11-24T06:18:33.780 INFO:teuthology.orchestra.run.smithi006.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T06:18:33.782 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-24T06:18:33.794 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-24T06:18:33.795 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-24T06:18:33.810 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T06:18:34.096 DEBUG:teuthology.orchestra.run.smithi006:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T06:18:34.382 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:18:34.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:18:34.499 DEBUG:teuthology.orchestra.run.smithi006:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T06:18:34.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:18:34.615 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-24T06:18:34.626 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-24T06:18:34.641 INFO:teuthology.task.internal:Duration was 984.360045 seconds 2023-11-24T06:18:34.641 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-24T06:18:34.660 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-24T06:18:34.661 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T06:18:34.671 DEBUG:teuthology.orchestra.run.smithi006:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T06:18:34.846 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:18:34.851 INFO:teuthology.orchestra.run.smithi006.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T06:18:35.278 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-24T06:18:35.278 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2023-11-24T06:18:35.279 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T06:18:35.347 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi006.front.sepia.ceph.com 2023-11-24T06:18:35.347 DEBUG:teuthology.orchestra.run.smithi006:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T06:18:35.432 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-24T06:18:35.433 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T06:18:35.443 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T06:18:35.545 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-24T06:18:35.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T06:18:35.551 DEBUG:teuthology.orchestra.run.smithi006:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T06:18:35.741 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-24T06:18:35.755 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-24T06:18:35.756 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T06:18:35.861 DEBUG:teuthology.orchestra.run.smithi006:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T06:18:35.964 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-24T06:18:35.975 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:18:35.985 DEBUG:teuthology.orchestra.run.smithi006:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:18:36.098 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2023-11-24T06:18:36.107 INFO:teuthology.orchestra.run.smithi006.stdout:kernel.core_pattern = core 2023-11-24T06:18:36.167 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:18:36.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:18:36.227 DEBUG:teuthology.orchestra.run.smithi006:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T06:18:36.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T06:18:36.286 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-24T06:18:36.298 INFO:teuthology.task.internal:Transferring archived files... 2023-11-24T06:18:36.300 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552/remote/smithi005 2023-11-24T06:18:36.300 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T06:18:36.418 DEBUG:teuthology.misc:Transferring archived files from smithi006:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552/remote/smithi006 2023-11-24T06:18:36.418 DEBUG:teuthology.orchestra.run.smithi006:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T06:18:36.529 INFO:teuthology.task.internal:Removing archive directory... 2023-11-24T06:18:36.529 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T06:18:36.538 DEBUG:teuthology.orchestra.run.smithi006:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T06:18:36.597 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-24T06:18:36.609 INFO:teuthology.task.internal:Not uploading archives. 2023-11-24T06:18:36.609 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-24T06:18:36.621 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-24T06:18:36.621 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T06:18:36.630 DEBUG:teuthology.orchestra.run.smithi006:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T06:18:36.685 INFO:teuthology.orchestra.run.smithi005.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 06:18 /home/ubuntu/cephtest 2023-11-24T06:18:36.694 INFO:teuthology.orchestra.run.smithi006.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 06:18 /home/ubuntu/cephtest 2023-11-24T06:18:36.697 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-24T06:18:36.710 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-24T06:18:36.776 INFO:teuthology.nuke:Checking targets against current locks 2023-11-24T06:18:36.868 DEBUG:teuthology.nuke:shortname: smithi005 2023-11-24T06:18:36.869 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:18:36.893 DEBUG:teuthology.nuke:shortname: smithi006 2023-11-24T06:18:36.894 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T06:18:36.897 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 05:53:28.331729', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHxC54AoB40hHZJBei+C+H1tj0i0mbvv+5ZiCTjBje1ZITrJ1meyYKiQ7HmT71SkrxFCLB1mDtonwt8WAVmum/Q='} 2023-11-24T06:18:36.919 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi006.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_05:41:33-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465552', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 05:53:28.329960', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMIKjSSSw3SKfP6l+THmyThcZ6hDVcLI75y1IghvDKxCpc9aPaHYHE7irCADEi8Q+VWGwhXLYcR4sptlmUvccwM='} 2023-11-24T06:18:36.946 INFO:teuthology.orchestra.console.smithi005:Power off 2023-11-24T06:18:36.947 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T06:18:36.970 DEBUG:teuthology.orchestra.console.smithi005:power off output: Chassis Power Control: Down/Off 2023-11-24T06:18:36.970 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:18:36.988 INFO:teuthology.orchestra.console.smithi006:Power off 2023-11-24T06:18:36.989 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T06:18:36.998 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2023-11-24T06:18:37.012 DEBUG:teuthology.orchestra.console.smithi006:power off output: Chassis Power Control: Down/Off 2023-11-24T06:18:37.012 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:18:37.036 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-11-24T06:18:40.999 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:18:41.038 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:18:41.108 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2023-11-24T06:18:41.147 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-11-24T06:18:45.109 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:18:45.147 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:18:45.219 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off 2023-11-24T06:18:45.259 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is on 2023-11-24T06:18:45.321 INFO:teuthology.orchestra.console.smithi005:Power off completed 2023-11-24T06:18:49.260 DEBUG:teuthology.orchestra.console.smithi006:pexpect command: ipmitool -H smithi006.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T06:18:49.371 DEBUG:teuthology.orchestra.console.smithi006:check power output: Chassis Power is off 2023-11-24T06:18:49.472 INFO:teuthology.orchestra.console.smithi006:Power off completed 2023-11-24T06:18:49.577 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/clone}} duration: 984.3600449562073 failure_reason: 'Test failure: test_clone_failure_status_failed (tasks.cephfs.test_volumes.TestSubvolumeSnapshotClones)' flavor: default mon.a-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 mon.b-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=3cff2a4b37ef43278fac27c72811efab status: fail success: false 2023-11-24T06:18:49.578 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T06:18:49.655 INFO:teuthology.run:FAIL